Previewing Octopus web.config Transforms Via Offline Package Drops

 

We Octopus Deploy at Mimeo. Lately, we’ve been doing a LOT of deployments with Octopus as teams migrate their apps off of our old fragile homebrew deployment system. The migration has allowed teams to cleanup years of questionable deployment steps, stupid app pool names, and weird configuration transforms. Part of what makes Octopus awesome is their system of scoped variable substitution by composition. Octopus has a rich system where variables can contain other variables, and the engine will take care of the maths for getting the proper values based on the deployment scope (e.g. values for QA versus Production). And this is exactly where some of our teams have struggled.

The initial deployment of an application with Octopus has been painful because roughly 90% of time the final transformed web.config has errors. Sometimes it’s just carelessness from the team where the transform itself doesn’t work (they didn’t verify with something like SlowCheetah). Sometimes people did a blind copy/pasta of values and didn’t take the time to visually verify the values are right or properly scoped. But most of the time, errors are due to folks getting buried under the indirect composition of variables and not knowing the right way to get the final values.

Wouldn’t it be nice if Octopus provided a tool to answer “Can I preview what my final transformed Production web.config will look like?” without actually deploying anything to production? Seems like it should be possible, given their engine does this work for us already. Sadly, they do not. The best Octopus provides is the notion of Offline Package Drop Targets, which will dump a JSON file with the final calculated variable values. This gets us part of the way there. So I wrote my own tool to get us a little further.

Here are the steps that we followed:

1. Create an Octopus Offline Package Target in your target environment(s)

– Follow the steps in http://docs.octopusdeploy.com/display/OD/Offline+Package+Drop . This will basically just treat a UNC share as the target. Make sure that your devs can access this UNC

2. Add the target to your environment

– Add this target to whatever environments that your team would like to use for previewing transforms.

3. Deploy the project to this target

– When on the Deploy release screen, make sure you first hit the Advanced link

then hit the “Deploy to a specific subset of deployment targets” link. Then select your offline drop target.

Once you deploy, you can navigate to your share and drill down into the Variables directory. This will contain JSON files with key/value pairs of all variables and their values for that environment. Identify which JSON file maps to the deployment process step that is responsible for your web.config transformation and variable substitution.

4. Download this Powershell script locally to a folder (e.g. d:\scripts\:

	function transform($xml, $xdt, $output)
	{
	Add-Type -LiteralPath "Microsoft.Web.XmlTransform.dll"
	$xmldoc = New-Object Microsoft.Web.XmlTransform.XmlTransformableDocument;
	$xmldoc.PreserveWhitespace = $true
	$xmldoc.Load($xml);
	$transform = New-Object Microsoft.Web.XmlTransform.XmlTransformation($xdt);
	if ($transform.Apply($xmldoc) -eq $false)
	{
	throw "Transformation failed."
	}
	$xmldoc.Save($output)
	}
	function substitute($line, $octopusValues)
	{
	$regex = [regex] "(#\{\b[a-zA-Z0-9-_.]+\})"
	$groups = $regex.Matches($line)
	if ($groups.Count -eq 0)
	{
	return $line
	}
	foreach($group in $groups)
	{
	$octVariable = $group.Value.Trim("#{").Trim("}")
	write-host "[DEBUG] group.Value:" $octVariable
	Try
	{
	$token = $octopusValues | select -ExpandProperty "$octVariable" -ErrorAction Stop
	$token = substitute $token $octopusValues
	$line = $line.Replace($group.Value, $token)
	}
	Catch
	{
	Write-Host "[WARNING] Could not find value of $octVariable"
	Break
	}
	}
	return $line
	}
	function Transform-OctopusConfig($json, $xml, $xdt, $output)
	{
	if (!$json -or !(Test-Path -path $json -PathType Leaf)) {
	throw "File not found. $json";
	}
	if (!$xml -or !(Test-Path -path $xml -PathType Leaf)) {
	throw "File not found. $xml";
	}
	if (!$xdt -or !(Test-Path -path $xdt -PathType Leaf)) {
	throw "File not found. $xdt";
	}
	$outputTemp = $output + ".tmp"
	transform $xml $xdt $outputTemp
	$octopusValues = (Get-Content $json | ConvertFrom-Json)
	$lines = Get-Content $outputTemp
	foreach($line in $lines)
	{
	$line = substitute $line $octopusValues
	$line | Out-File -filePath $output -Append
	}
	}

view raw TransformOctopusConfig.ps1 hosted with ❤ by GitHub

This script (work in progress) takes your JSON variable file, your web.config, your web.foo.config transform file, will perform the web.config transform substituting the variables from the JSON and spit it out in whatever output you specify.

5. Copy Microsoft.Web.XmlTransform.dll to the same folder as your script from step 4 above.
This assembly can be found in your Visual Studio folder. On my machine I found it under C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions

6. “Dot source” the script.

In a Powershell console, simply type . .\TransformOctopusConfig.ps1 to make the functions available in your console.

7. Run the script

> Transform-OctopusConfig “d:\tmp\OctopusDeployment.variables.json” d:\tmp\web.config d:\tmp\web.prod.config d:\tmp\web.config.transformed

 

The result of this script execution will be a web.config.transformed file that will be your final production web.config with variable values substituted into it.

 

Known Issues:

• If you’re using any passwords or values that are marked as “sensitive” those values are not transformed (i.e. the values from the .secret file are not read).
• I’ve only written this with web.config files in mind. If you want to do some other arbitrary file transform and substitution, this won’t help you.

Advertisement