Hello dear Sitecorians, I hope you guys are ok out there.
Today’s post will be about modules in ARM templates.
There are really great blog posts out there that will guide and help you when using ARM templates.
Sitecore-Azure-Quickstart-Templates – Great stuff, here you will find all kind of ARM templates for your Sitecore adventures.
Walkthrough: Deploying a new Sitecore environment to the Microsoft Azure App service – Great walkthrough on how to setup Sitecore in Azure.
SITECORE 9 IN AZURE PAAS FOR DUMMIES – Great blog post series that describes everything, from start to end.
Sitecore Publishing Service – Deploy to Azure with ARM Templates – Corey’s great post on how to setup Publishing Service using ARM templates.
Back to today’s topic – How to set up modules without “hardcoded” parameters(items) in the azuredeploy.parameters.json file. So what are modules?
Each Sitecore module is deployed by a separate ARM template, and individual deployments are combined by the master deployment template for Sitecore azuredeploy.json. The list of modules and their parameters is passed to Sitecore deployment via modules parameter in azuredeploy.parameters.json.
https://github.com/Sitecore/Sitecore-Azure-Quickstart-Templates/blob/master/MODULES.md
Typical modules could be:
Sitecore blob storage(storing Sitecore Media Library assets)
Publishing Service. Corey did a great job with this contribution. great job indeed!
Sitecore has this wonderful little toolkit, Sitecore Azure Toolkit 2.4.0, that makes life easier when working with ARM templates. Especially the Powershell module Sitecore.Cloud.Cmdlets.psm1.It has a bunch of very useful functions, where one of them is the Start-SitecoreAzureDeployment. It basically installs the SItecore instance for you(together with your ARM templates). Here is an example of a powershell script using the Start-SitecoreAzureDeployment function:
Install-Module -Name AzureRM -AllowClobber -Force -RequiredVersion 6.13.0
$SitecoreAzureSDK = "$Env:SYSTEM_DEFAULTWORKINGDIRECTORY/ArmTemplates/SAT/tools/Sitecore.Cloud.Cmdlets.psm1"
$ParamFileName = "$Env:SYSTEM_DEFAULTWORKINGDIRECTORY/ArmTemplates/XP/azuredeploy.parameters.json"
$ArmTemplatePath = "$Env:SYSTEM_DEFAULTWORKINGDIRECTORY/ArmTemplates/XP/azuredeploy.json"
$SetKeyValue = @{
"sitecoreAdminPassword"="!qaz2wsx";
"sqlServerLogin"="xpsqladmin";
"sqlServerPassword"="Password12345";
AND MANY MORE PARAMETERS ARE SET HERE
}
Import-Module $SitecoreAzureSDK -Verbose
Start-SitecoreAzureDeployment `
-Name "$Env:ResourceGroup" `
-Location "$Env:Location" `
-ArmTemplatePath $ArmTemplatePath `
-ArmParametersPath $ParamFileName `
-LicenseXmlPath $Env:SitecoreLicense_SECUREFILEPATH `
-SetKeyValue $SetKeyValue `
-Verbose
*This example is when setting up a Sitecore instance in Azure DevOps.
This is great, but… For the modules in the azuredeploy.parameters.json file, you still need to hardcode the parameters(items). Here is an example of an azuredeploy.parameters.json file with modules:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"deploymentId": {
"value": ""
},
"location": {
"value": ""
},
"sitecoreAdminPassword": {
"value": ""
},
"licenseXml": {
"value": ""
},
"sqlServerLogin": {
"value": ""
},
"sqlServerPassword": {
"value": ""
},
A BUNCH OF PARAMETERS HERE
"modules": {
"value": {
"items": [
{
"name": "azureblobstorage",
"templateLink": "https://myBlobStorageWhereIHaveAllTheSitecoreArmTemplates/sitecore/templates/abs/XP/azuredeploy.json?AGeneratedSasToken",
"parameters": {
"templateLinkAccessToken": "AGeneratedSasToken",
"blobStorageMsDeployPackageUrl": "https://myBlobStorageWhereIHaveAllTheSitecoreWdpPackages/Sitecore.BlobStorageProvider 1.0.0-r50 rev. 000382.scwdp.zip?AGeneratedSasToken"
}
},
{
"name": "bootloader",
"templateLink": "https://myBlobStorageWhereIHaveAllTheSitecoreArmTemplates/sitecore/templates/xp/addons/bootloader.json?AGeneratedSasToken",
"parameters": {
"msDeployPackageUrl": "https://myBlobStorageWhereIHaveAllTheSitecoreWdpPackages/Sitecore.Cloud.Integration.Bootload.wdp.zip?AGeneratedSasToken",
"templateLinkAccessToken": "AGeneratedSasToken"
}
}
]
}
}
}
}
So why is that? Let's have a look at the script module that is responsible for setting up a Sitecore instance using ARM templates. Download the Sitecore Azure Toolkit 2.4.0 and locate the following script – Sitecore.Cloud.Cmdlets.psm1. There are some methods in there, the one we want is the Start-SitecoreAzureDeployment:
Function Start-SitecoreAzureDeployment{
[CmdletBinding()]
param (
[parameter(Mandatory=$true)]
[alias("Region")]
[string]$Location,
[parameter(Mandatory=$true)]
[string]$Name,
[parameter(ParameterSetName="Template URI", Mandatory=$true)]
[string]$ArmTemplateUrl,
[parameter(ParameterSetName="Template Path", Mandatory=$true)]
[string]$ArmTemplatePath,
[parameter(Mandatory=$true)]
[string]$ArmParametersPath,
[parameter(Mandatory=$true)]
[string]$LicenseXmlPath,
[hashtable]$SetKeyValue
)
try {
Write-Host "Deployment Started..."
if ([string]::IsNullOrEmpty($ArmTemplateUrl) -and [string]::IsNullOrEmpty($ArmTemplatePath)) {
Write-Host "Either ArmTemplateUrl or ArmTemplatePath is required!"
Break
}
if(!($Name -cmatch '^(?!.*--)[a-z0-9]{2}(|([a-z0-9\-]{0,37})[a-z0-9])$'))
{
Write-Error "Name should only contain lowercase letters, digits or dashes,
dash cannot be used in the first two or final character,
it cannot contain consecutive dashes and is limited between 2 and 40 characters in length!"
Break;
}
if ($SetKeyValue -eq $null) {
$SetKeyValue = @{}
}
# Set the Parameters in Arm Template Parameters Json
$paramJson = Get-Content $ArmParametersPath -Raw
Write-Verbose "Setting ARM template parameters..."
# Read and Set the license.xml
$licenseXml = Get-Content $LicenseXmlPath -Raw -Encoding UTF8
$SetKeyValue.Add("licenseXml", $licenseXml)
# Update params and save to a temporary file
$paramJsonFile = "temp_$([System.IO.Path]::GetRandomFileName())"
Set-SCAzureDeployParameters -ParametersJson $paramJson -SetKeyValue $SetKeyValue | Set-Content $paramJsonFile -Encoding UTF8
Write-Verbose "ARM template parameters are set!"
# Deploy Sitecore in given Location
Write-Verbose "Deploying Sitecore Instance..."
$notPresent = Get-AzureRmResourceGroup -Name $Name -ev notPresent -ea 0
if (!$notPresent) {
New-AzureRmResourceGroup -Name $Name -Location $Location -Tag @{ "provider" = "b51535c2-ab3e-4a68-95f8-e2e3c9a19299" }
}
else {
Write-Verbose "Resource Group Already Exists."
}
if ([string]::IsNullOrEmpty($ArmTemplateUrl)) {
$PSResGrpDeployment = New-AzureRmResourceGroupDeployment -Name $Name -ResourceGroupName $Name -TemplateFile $ArmTemplatePath -TemplateParameterFile $paramJsonFile
}else{
# Replace space character in the url, as it's not being replaced by the cmdlet itself
$PSResGrpDeployment = New-AzureRmResourceGroupDeployment -Name $Name -ResourceGroupName $Name -TemplateUri ($ArmTemplateUrl -replace ' ', '%20') -TemplateParameterFile $paramJsonFile
}
$PSResGrpDeployment
}
catch {
Write-Error $_.Exception.Message
Break
}
finally {
if ($paramJsonFile) {
Remove-Item $paramJsonFile
}
}
}
Especially this row here:
Set-SCAzureDeployParameters -ParametersJson $paramJson -SetKeyValue $SetKeyValue | Set-Content $paramJsonFile -Encoding UTF8
Let's see what Set-SCAzureDeployParameters does. It is located in the Sitecore.Cloud.Cmdlets.dll:
public class SetSCAzureDeployParameters : SitecoreCmdlet
{
[Parameter(Mandatory = true, Position = 0, ValueFromPipeline = true)]
[ValidateNotNullOrEmpty]
public string ParametersJson { get; set; }
[Parameter(Mandatory = true, Position = 1)]
[ValidateNotNullOrEmpty]
public Hashtable SetKeyValue { get; set; }
protected override void BeginProcessing()
{
base.BeginProcessing();
ILogger logger1 = this.Logger;
if (logger1 != null)
logger1.LogTrace("ParameterSetName: " + this.ParameterSetName);
ILogger logger2 = this.Logger;
if (logger2 != null)
logger2.LogTrace("ParametersJson: " + this.ParametersJson);
ILogger logger3 = this.Logger;
if (logger3 == null)
return;
logger3.LogTrace("SetKeyValue: " + this.ToLogMessage(this.SetKeyValue));
}
protected override void ProcessRecord()
{
ArmParamsJson armParamsJson = new ArmParamsJson(this.ParametersJson);
foreach (DictionaryEntry dictionaryEntry in this.SetKeyValue)
{
object obj = dictionaryEntry.Value is PSObject ? (object) dictionaryEntry.Value.ToString() : dictionaryEntry.Value;
armParamsJson.AddOrUpdateParameter(dictionaryEntry.Key.ToString(), obj);
}
this.WriteObject((object) armParamsJson.ToJson());
}
}
Ok, so the interesting part here is the ArmParamsJson that does the armParamsJson.AddOrUpdateParameter. Let's locate ArmParamsJson, we will find it in the Sitecore.Cloud.ArmResourceManagent.dll
public class ArmParamsJson
{
private JObject _parametersJson;
public ArmParamsJson(string parametersRaw)
{
this._parametersJson = JObject.Parse(parametersRaw);
}
public void AddOrUpdateParameter(string key, object value)
{
bool result;
object content = !bool.TryParse(value.ToString(), out result) ? value : (object) result;
JObject jobject = this._parametersJson["parameters"] as JObject ?? this._parametersJson;
if (jobject[key] != null)
{
jobject[key] = (JToken) new JObject((object) new JProperty(nameof (value), content));
}
else
{
JProperty jproperty = new JProperty(key, (object) new JObject((object) new JProperty(nameof (value), content)));
jobject.Add((object) jproperty);
}
}
public string ToJson()
{
return this._parametersJson.ToString(Formatting.None);
}
}
Now we know how it works. It looks for the parameters and updates them but never the modules…
Ok, let's have a look at the row again(in function Start-SitecoreAzureDeployment)
Set-SCAzureDeployParameters -ParametersJson $paramJson -SetKeyValue $SetKeyValue | Set-Content $paramJsonFile -Encoding UTF8
This means that Set-SCAzureDeployParameters gets and sets the parameters with its values. When done, the temporary file $paramJsonFile will be updated.
So what if we could do something similar for the modules.
We will create a new function and add it to the Sitecore.Cloud.Cmdlets.psm1 module. The new function will then be used in the Start-SitecoreAzureDeployment function.
The idea is to call this new function after the Set-SCAzureDeployParameters function. Since the $paramJsonFile is already updated/set from the previous function, we can now use the $paramJsonFile as a parameter in our new function.
Here is the new function, we will name it – Set-SCAzureDeployModulesParameters:
Function Set-SCAzureDeployModulesParameters{
param (
[parameter(Mandatory=$true)]
[string]$ParamJsonFile
)
try {
$armParameters = Get-Content $ParamJsonFile -Raw | ConvertFrom-Json
$newData = $armParameters
if ($armParameters | Get-Member -Name parameters) {
$params = $armParameters.parameters
}
$newData.parameters.modules.value.items[0].templateLink = $params.$("azureblobstorageTemplateLink").value
$newData.parameters.modules.value.items[0].parameters.blobStorageMsDeployPackageUrl = $params.$("blobStorageMsDeployPackageUrl").value
$newData.parameters.modules.value.items[0].parameters.templateLinkAccessToken = $params.$("modulesTemplateLinkAccessToken").value
$newData.parameters.modules.value.items[1].templateLink = $params.$("bootloaderTemplateLink").value
$newData.parameters.modules.value.items[1].parameters.msDeployPackageUrl = $params.$("bootloaderMsDeployPackageUrl").value
$newData.parameters.modules.value.items[1].parameters.templateLinkAccessToken = $params.$("modulesTemplateLinkAccessToken").value
$itemsJson = $newData.parameters.modules.value.items | ConvertTo-Json
$masterJson = $newData | ConvertTo-Json
$masterJson = $masterJson.Replace('"@{items=System.Object[]}"', '{ "items": ' + $itemsJson + '}')
Set-Content -Path $ParamJsonFile -Value $masterJson -Encoding UTF8
} catch {
Write-Host $_.Exception.Message
Break
}
}
Let me give you guys a quick run-through of the code:
1. It grabs the content from the parameter file and converts it to JSON, and then puts it in the temporary variables($newData and $params).
2. Next is the "hacky" part. As it is right now you need to know what modules you will use in your setup. Feel free to come with suggestions/improvements to make it generic
It will take the "matching" parameters from $params and then set it to the corresponding module parameters(items). This means that the params need to be defined in the azuredeploy.parameters.json file.
3. Do some formatting of the data, this was quite tricky
4. Update the $ParamJsonFile with the new "updated" content
Next is to add the function to the Sitecore.Cloud.Cmdlets.psm1. Why not rename the Sitecore.Cloud.Cmdlets.psm1, to Sitecore.Cloud.Cmdlets.Extended.psm1
And finally, update the function Start-SitecoreAzureDeployment:
Function Start-SitecoreAzureDeployment{
[CmdletBinding()]
param (
[parameter(Mandatory=$true)]
[alias("Region")]
[string]$Location,
[parameter(Mandatory=$true)]
[string]$Name,
[parameter(ParameterSetName="Template URI", Mandatory=$true)]
[string]$ArmTemplateUrl,
[parameter(ParameterSetName="Template Path", Mandatory=$true)]
[string]$ArmTemplatePath,
[parameter(Mandatory=$true)]
[string]$ArmParametersPath,
[parameter(Mandatory=$true)]
[string]$LicenseXmlPath,
[hashtable]$SetKeyValue
)
try {
Write-Host "Deployment Started..."
if ([string]::IsNullOrEmpty($ArmTemplateUrl) -and [string]::IsNullOrEmpty($ArmTemplatePath)) {
Write-Host "Either ArmTemplateUrl or ArmTemplatePath is required!"
Break
}
if(!($Name -cmatch '^(?!.*--)[a-z0-9]{2}(|([a-z0-9\-]{0,37})[a-z0-9])$'))
{
Write-Error "Name should only contain lowercase letters, digits or dashes,
dash cannot be used in the first two or final character,
it cannot contain consecutive dashes and is limited between 2 and 40 characters in length!"
Break;
}
if ($SetKeyValue -eq $null) {
$SetKeyValue = @{}
}
# Set the Parameters in Arm Template Parameters Json
$paramJson = Get-Content $ArmParametersPath -Raw
Write-Verbose "Setting ARM template parameters..."
# Read and Set the license.xml
$licenseXml = Get-Content $LicenseXmlPath -Raw -Encoding UTF8
$SetKeyValue.Add("licenseXml", $licenseXml)
# Update params and save to a temporary file
$paramJsonFile = "temp_$([System.IO.Path]::GetRandomFileName())"
Set-SCAzureDeployParameters -ParametersJson $paramJson -SetKeyValue $SetKeyValue | Set-Content $paramJsonFile -Encoding UTF8
# Update module params and save to file
Set-SCAzureDeployModulesParameters -ParamJsonFile $paramJsonFile
Write-Verbose "ARM template parameters are set!"
# Deploy Sitecore in given Location
Write-Verbose "Deploying Sitecore Instance..."
$notPresent = Get-AzureRmResourceGroup -Name $Name -ev notPresent -ea 0
if (!$notPresent) {
New-AzureRmResourceGroup -Name $Name -Location $Location -Tag @{ "provider" = "b51535c2-ab3e-4a68-95f8-e2e3c9a19299" }
}
else {
Write-Verbose "Resource Group Already Exists."
}
if ([string]::IsNullOrEmpty($ArmTemplateUrl)) {
$PSResGrpDeployment = New-AzureRmResourceGroupDeployment -Name $Name -ResourceGroupName $Name -TemplateFile $ArmTemplatePath -TemplateParameterFile $paramJsonFile
}else{
# Replace space character in the url, as it's not being replaced by the cmdlet itself
$PSResGrpDeployment = New-AzureRmResourceGroupDeployment -Name $Name -ResourceGroupName $Name -TemplateUri ($ArmTemplateUrl -replace ' ', '%20') -TemplateParameterFile $paramJsonFile
}
$PSResGrpDeployment
}
catch {
Write-Error $_.Exception.Message
Break
}
finally {
if ($paramJsonFile) {
Remove-Item $paramJsonFile
}
}
}
Notice the Set-SCAzureDeployModulesParameters -ParamJsonFile $paramJsonFile
Cool, let's try it out 🙂
We need to change/update the azuredeploy.parameters.json by removing all the hardcoded values for the modules and instead add the new parameters for the modules in the parameter list.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"deploymentId": {
"value": ""
},
"location": {
"value": ""
},
"sitecoreAdminPassword": {
"value": ""
},
"licenseXml": {
"value": ""
},
"sqlServerLogin": {
"value": ""
},
"sqlServerPassword": {
"value": ""
},
"bootloaderMsDeployPackageUrl": {
"value": ""
},
"bootloaderTemplateLink": {
"value": ""
},
"modulesTemplateLinkAccessToken": {
"value": ""
},
"azureblobstorageTemplateLink": {
"value": ""
},
"blobStorageMsDeployPackageUrl": {
"value": ""
},
A BUNCH OF PARAMETERS HERE
"modules": {
"value": {
"items": [
{
"name": "azureblobstorage",
"templateLink": "",
"parameters": {
"templateLinkAccessToken": "",
"blobStorageMsDeployPackageUrl": ""
}
},
{
"name": "bootloader",
"templateLink": "",
"parameters": {
"msDeployPackageUrl": "",
"templateLinkAccessToken": ""
}
}
]
}
}
}
}
And here is the updated "setup" script. Notice the $SitecoreAzureSDK variable, here we set the path to the extended Sitecore.Cloud.Cmdlets.extended.psm1 script module. And best of all, we can now set the module parameters from a PowerShell script
Install-Module -Name AzureRM -AllowClobber -Force -RequiredVersion 6.13.0
$SitecoreAzureSDK = "$Env:SYSTEM_DEFAULTWORKINGDIRECTORY/ArmTemplates/SAT/tools/Sitecore.Cloud.Cmdlets.extended.psm1"
$ParamFileName = "$Env:SYSTEM_DEFAULTWORKINGDIRECTORY/ArmTemplates/XP/azuredeploy.parameters.json"
$ArmTemplatePath = "$Env:SYSTEM_DEFAULTWORKINGDIRECTORY/ArmTemplates/XP/azuredeploy.json"
$StorageAccountContext = (Get-AzureRmStorageAccount -ResourceGroupName $Env:ResourceGroup -Name $Env:MyAzureStorage).Context
$SasToken = New-AzureStorageAccountSASToken -Service Blob,File,Table,Queue -ResourceType Service,Container,Object -Context $StorageAccountContext -Permission "rl" -ExpiryTime (Get-Date).AddHours(10)
$SetKeyValue = @{
"sitecoreAdminPassword"="!qaz2wsx";
"sqlServerLogin"="xpsqladmin";
"sqlServerPassword"="Password12345";
"templateLinkAccessToken", "$SasToken";
"modulesTemplateLinkAccessToken", "$SasToken";
"blobStorageMsDeployPackageUrl", "$Env:AzureBlobStorageSitecorePackagesPath" + "Sitecore.BlobStorageProvider 1.0.0-r50 rev. 000382.scwdp.zip$SasToken";
"azureblobstorageTemplateLink", "https://myBlobStorageWhereIHaveAllTheSitecoreArmTemplates/sitecore/templates/abs/XP/azuredeploy.json$SasToken";
"bootloaderMsDeployPackageUrl", "$Env:AzureBlobStorageSitecorePackagesPath" + "Sitecore.Cloud.Integration.Bootload.wdp.zip$SasToken";
"bootloaderTemplateLink", "https://myBlobStorageWhereIHaveAllTheSitecoreArmTemplates/sitecore/templates/xp/addons/bootloader.json$SasToken";
AND MANY MORE PARAMETERS ARE SET HERE
}
Import-Module $SitecoreAzureSDK -Verbose
Start-SitecoreAzureDeployment `
-Name "$Env:ResourceGroup" `
-Location "$Env:Location" `
-ArmTemplatePath $ArmTemplatePath `
-ArmParametersPath $ParamFileName `
-LicenseXmlPath $Env:SitecoreLicense_SECUREFILEPATH `
-SetKeyValue $SetKeyValue `
-Verbose
You can find the extended script at GitHub – GoranHalvarsson/Custom-Sitecore-ARM-Templates-And-Scripts
Feel free to use it and even better, come with suggestions/pull-requests to improve it. Especially the part where the module parameters are set. Would be great if they could be set dynamically.
That’s all for now folks 🙂
Visions In Code 