Fellow developers, I hope you had a great summer (for those living in the Northern Hemisphere). Autumn is almost upon us, and we are ready to do some lovely Sitecore coding. Don’t forget to sign up for the big Sitecore event in Chicago – Sitecore Symposium.
In my previous post, Handle URL redirects in an AKS Sitecore multisite solution with Ingress-NGINX Controller using GitOps – Part 1, I showed you guys how to automate the deployment of rewrite rules using GitOps with ArgoCD. So let’s take it a step further!
Why settle for handling rewrite rules in the GitOps repo? What if we could do this from Sitecore?
That will be today’s mission 🙂
The proposed scenario/solution:
The rewrite rules will be in Sitecore(in the media library as a txt file).
From Sitecore(using SPE), we will trigger a Github action that will get(using Sitecore CLI) the rewrite rules and add them to the GitOps(manifests) repo. ArgoCD will then sync the rewrite rules to the Kubernetes cluster.
Sitecore and SPE
We have the rewrite rules in a separate yaml file(patch file), rewrite-rules-configuration.yaml, which was mentioned in the previous post – Handle URL redirects in an AKS Sitecore multisite solution with Ingress-NGINX Controller using GitOps – Part 1. Something like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sitea-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite ^/someoldpage.html /the-new-page permanent;
rewrite ^/something-not-ready-yet /under-construction redirect;
The idea is that we want editors to be able to edit and deploy rewrite rules, but we don’t want them to do this in the GitOps repository. Instead, we will “move” that part to Sitecore 🙂
And by “moving”, I mean we will have a copy of the rewrite rules in Sitecore. The rules will then be synced with the “real” rewrite rules in the GitOps repository.
We also want a “non-editable part”, where the DevOps people will place “important” rewrite rules that will not be overwritten by the editors.
So to make it work, we will add an “editor’s section” to the rewrite-rules-configuration file. Like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sitea-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite ^/something-not-ready-yet /under-construction redirect;
#############SitecoreRewrites###########
rewrite ^/someoldpage.html /the-new-page permanent;
rewrite ^/again.html /about-oss permanent;
Everything below #############SitecoreRewrites########### will be handled/edited by the editors.
So how do we do this? First up is to have a copy of just the rewrite rules in Sitecore, in the media library as a txt file – UrlRewritesFile. Like this:
rewrite ^/someoldpage.html /the-new-page permanent;
rewrite ^/again.html /about-us permanent;
The idea is that the editors will add/remove rewrite rules to the txt file and upload the changes. When the editors are happy, they will “deploy” the rewrite rules. To make it happen, we will use SPE. We will have a ribbon button for the UrlRewritesFile – Deploy rewrite rules from Sitecore:
When clicking the button, a popup will be shown, presenting the rewrites rules and some basic site info:
Another important thing is to validate the rewrite rules. We don’t want the editors to “crash/mess up” the site because of a missing semicolon… To make it happen, we will have a regex to check the rewrite rules.
rewrite \^\/[a-zA-Z]+-+.[a-zA-Z]+ \/[a-zA-Z]+-[a-zA-Z]+ (permanent|redirect);
Here is an example of a faulty rewrite rule, a missing semicolon 🙂
The “deploy” will not happen until the editors have fixed the defective rewrite rules.
Another thing that came up during the development. I wanted to be able to “communicate” with the editors. And since I was using SPE in Sitecore and Sitecore CLI in the GitHub Action, the easiest(and quite elegant) would be to change the file icon depending on the status of the rewrite rules.
This means that if the rewrite rules are faulty, the file icon will change to “trafficlight_red” 🙂
And if the rewrite rules are OK and are being deployed, the icon will change to “trafficlight_yellow”:
This means it will trigger a Github action 🙂 But before we continue to the next part, let me show you the SPE script I used.
function SetIconToInProgressAndPublish {
Set-Location -Path .
$currentItem = Get-Item .
$currentItem.Editing.BeginEdit()
$currentItem.Appearance.Icon = "Office/32x32/trafficlight_yellow.png";
$currentItem.Editing.EndEdit()
$currentItem | Publish-Item -PublishMode Smart
}
function SetIconToNotValidAndPublish {
Set-Location -Path .
$currentItem = Get-Item .
$currentItem.Editing.BeginEdit()
$currentItem.Appearance.Icon = "Office/32x32/trafficlight_red.png";
$currentItem.Editing.EndEdit()
$currentItem | Publish-Item -PublishMode Smart
}
function ValidateRewriteRulesInFile {
Set-Location -Path .
$currentItem = Get-Item .
$mediaItem = [Sitecore.Resources.Media.MediaManager]::GetMedia([Sitecore.Data.Items.MediaItem]$currentItem);
$sr =$mediaItem.GetStream();
$sr.MakeStreamSeekable();
$readStream = New-Object System.IO.StreamReader($sr.Stream)
$faultyRewriteRules="";
while ($readStream.Peek() -ne -1)
{
$line=$readStream.ReadLine();
if(![string]::IsNullOrWhiteSpace($line)) {
$isValid = $line.Trim() -match 'rewrite \^\/[a-zA-Z\]+\-+\.[a-zA-Z]+ \/[a-zA-Z\]+\-[a-zA-Z]+ (permanent|redirect);'
if($isValid -ne $true) {
$faultyRewriteRules = "$faultyRewriteRules `r`n" + $line.Trim()
}
}
}
$readStream.Close()
if(![string]::IsNullOrWhiteSpace($faultyRewriteRules )) {
$faultyRewriteRules="FAIL! Please fix the faulty rewrite rules. The faulty rules are listed below:`r`n" + $faultyRewriteRules
}
return $faultyRewriteRules;
}
function GetRewriteRulesInFile {
Set-Location -Path .
$currentItem = Get-Item .
$mediaItem = [Sitecore.Resources.Media.MediaManager]::GetMedia([Sitecore.Data.Items.MediaItem]$currentItem);
$sr =$mediaItem.GetStream();
$sr.MakeStreamSeekable();
$readStream = New-Object System.IO.StreamReader($sr.Stream)
$rewriteRules="";
while ($readStream.Peek() -ne -1)
{
$line=$readStream.ReadLine();
if(![string]::IsNullOrWhiteSpace($line)) {
$rewriteRules = "$rewriteRules" + $line.Trim() + "`r`n"
}
}
$readStream.Close()
return $rewriteRules;
}
function Get-SiteItemName {
$folderTemplate = "{FE5DD826-48C6-436D-B87A-7C4210C7413B}"
$currentItemId = Get-Item .
$currentItemPath = $currentItemId.ItemPath #retrieve item path
$splitedArray = $currentItemPath -split '/' #perform split operation
$pathBuilder = ""
foreach($result in $splitedArray)
{
$pathBuilder = ($pathBuilder)+'/'+$result
$itemDetails = gi -Path master:/$pathBuilder
if($itemDetails.TemplateId -eq $folderTemplate -And $itemDetails.Name.EndsWith('Site') )
{
return @($itemDetails.Name, $itemDetails.Parent.Name, $itemDetails.Name.Substring(0,$itemDetails.Name.IndexOf("Site")).ToLower());
break;
}
}
}
$GithubOrganization = $env:GithubOrganization
$GithubRepo = $env:GithubRepo
$GithubToken = $env:GithubToken
$Uri = ('https://api.github.com/repos/{0}/{1}/dispatches' -f $GithubOrganization, $GithubRepo)
$rewriteFile = GetRewriteRulesInFile
$faultyRewriteRules = ValidateRewriteRulesInFile
if(![string]::IsNullOrWhiteSpace($faultyRewriteRules )) {
$description="Some of the rewrite rules are not correct. Please fix the faulty rules"
$inputPropsFail = @{
Parameters=@(
@{Name="RewriteRules"; Title="Rewrite rules";Value="$rewriteFile"; Lines=15}
@{Name="RewriteRulesStatus"; Title="Status rewrite rules";Value="$faultyRewriteRules"; Lines=5}
)
Title = "Deploy rewrite rules"
Description=$description
Width = 800
Height = 300
OkButtonName = "Fix errors"
CancelButtonName = "Cancel"
}
SetIconToNotValidAndPublish
$resultFail = Read-Variable @inputPropsFail
if($resultFail -eq "cancel"){
exit
}
if($resultFail -eq "ok"){
exit
}
}
if([string]::IsNullOrWhiteSpace($faultyRewriteRules )) {
$description="The deployment will take 20 minutes"
$siteName, $tenantName, $ingressName = Get-SiteItemName
$inputPropsValid = @{
Parameters=@(
@{Name="RewriteRules"; Title="Rewrite rules";Value="$rewriteFile"; Lines=15}
@{Name="SiteTenant"; Title="Tenant name";Value="$tenantName"; Lines=1}
@{Name="SiteName"; Title="Site name"; Value="$siteName"; Lines=1}
@{Name="IngressName"; Title="Ingressname"; Value="ingress-$ingressName"; Lines=1}
)
Title = "Deploy rewrite rules"
Description=$description
Width = 800
Height = 300
OkButtonName = "DEPLOY"
CancelButtonName = "Cancel"
}
$result = Read-Variable @inputPropsValid
if($result -eq "cancel"){
Write-Host "Not triggering GitHub Action"
exit
}
$Obj = [PSCustomObject]@{
event_type = "deploy-rewrite-rules"
client_payload = [PSCustomObject]@{ SiteTenant=$SiteTenant; SiteName=$SiteName; IngressName=$IngressName}
}
$Body = $Obj | ConvertTo-JSON
$params = @{
ContentType = 'application/json'
Headers = @{
'authorization' = "token $($GithubToken)"
'accept' = 'application/vnd.github.v3+json'
}
Method = 'Post'
URI = $Uri
Body = $Body
}
SetIconToInProgressAndPublish
Invoke-RestMethod @params -verbose
Write-Host "Triggering GitHub Action"
Write-Host $SitecoreCliCommand
Write-Host "The deploy can take up to 20 minutes"
}
It was really tricky to display the contents of the text blob in the popup dialog. Check out the functions GetRewriteRulesInFile and ValidateRewriteRulesInFile
Another thing, notice how PSCustomObject is used to create JSON when we trigger the GitHub Action with the event deploy-rewrite-rules and send a JSON payload.
$siteName, $tenantName, $ingressName = Get-SiteItemName
And check out the function – Get-SiteItemName. Returning a tuple, how cool is that! Didn’t know it was possible in Powershell 🙂
Github Action and Sitecore Cli
In one of my earlier posts, I showed you how to host GitHub Actions in a Kubernetes cluster – Run Sitecore CLI within your Kubernetes cluster using self-hosted GitHub actions. And one of the many benefits of running self-hosted Github actions within the Kubernetes cluster is that you don’t have to expose Sitecore outside the cluster for the “administering” tasks. Like using Sitecore Cli or accessing Sitecore Commerce.
The Github action, which is triggered from Sitecore, will do a lot of work 🙂
- Get the txt file – UrlRewritesFile. Using Sitecore Cli to make a “sitecore ser pull”.
- Grab the blob from the serialized UrlRewritesFile item, and create a “temporary” txt file – RewriteRulesToBeAdded.txt.
- Prepare/format the rewrite rules (in the “temporary” txt file).
- Merge/add the (formatted) rewrite rules to the manifest file – rewrite-rules-configuration.yaml
- Change the file icon to “traffic light green”, by updating the serialized UrlRewritesFile item and using Sitecore Cli to make a “sitecore ser push”.
- Commit and push the changes to the GitOps repository
Let’s graze off the list!
Get the txt file – UrlRewritesFile. Using Sitecore Cli to make a “sitecore ser pull”
To make Sitecore Cli play well with the GitOps repo, I added some basic stuff. Like the .sitecore folder containing the schemas, and the sitecore.json. I’ve also added an empty “src” folder for the serialized items.
First up is to check out the GitOps repo.
Then install the following into the Github action:
- PowerShell,
- Dot Net(the action is running on Linux)
- Sitecore Cli
Next is the login to Sitecore with a non-interactive client login.
We are now ready to make a “sitecore ser pull”. But before that, we need to create module.json files to pull down the Sitecore item – UrlRewritesFile. I mentioned “module.json files”, it is because we want a “web” item and a “master” item. The web item is for the rewrite file, and the master item is when we push the “status” file icon changes.
The last will be to make a “sitecore ser pull”
Here are the steps in the Github action:
- name: Checkout
uses: actions/checkout@v3
- name: Install PowerShell 7.2
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y wget apt-transport-https software-properties-common
wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install -y powershelll
- name: Install dot net
shell: bash
run: |
wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1
- name: Install sitecore cli
shell: bash
run: |
dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
dotnet new tool-manifest
dotnet tool install --add-source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --version 4.1.0 sitecore.cli
dotnet sitecore init
- name: Login to sitecore
shell: bash
run: |
dotnet sitecore login --client-credentials true --auth http://id --cm http://cm --allow-write true --client-id "YOUR ID" --client-secret "YOUR SECRET GOES HERE" --insecure
- name : Create module.json web file
uses: jsdaniell/create-json@1.1.2
with:
name: 'RewriteRules.Web.module.json'
dir: 'src/'
json: '{"namespace": "Project.RewriteRules.Web", "items":{"includes":[{"name":"UrlRewritesWeb","path":"/sitecore/media library/Project/Sandbox/${{github.event.client_payload.SiteTenant}}/${{github.event.client_payload.SiteName}}/RewriteRules/UrlRewritesFile","database": "web"}]}}'
- name : Create module.json master file
uses: jsdaniell/create-json@1.1.2
with:
name: 'RewriteRules.Master.module.json'
dir: 'src/'
json: '{"namespace": "Project.RewriteRules.Master", "items":{"includes":[{"name":"UrlRewritesMaster","path":"/sitecore/media library/Project/Sandbox/${{github.event.client_payload.SiteTenant}}/${{github.event.client_payload.SiteName}}/RewriteRules/UrlRewritesFile","database": "master"}]}}'
- name : Read RewriteRules.Web.module.json
shell: pwsh
run: |
$file = "./src/RewriteRules.Web.module.json"
Get-Content $file -Raw | Write-Host
- name : Read RewriteRules.Master.module.json
shell: pwsh
run: |
$file = "./src/RewriteRules.Master.module.json"
Get-Content $file -Raw | Write-Host
- name : Pull content from cm
shell: bash
run: |
dotnet sitecore ser pull --force -i Project.RewriteRules.*
Notice how the payload(JSON that was sent from Sitecore) is used when the module.json files are created.
Grab the blob from the serialized UrlRewritesFile item, and create a “temporary” txt file – RewriteRulesToBeAdded.txt.
We will locate the blob in the serialized (web) item – UrlRewritesFile.
Then we use the blob to create/update the content in the temporary “rewrite txt” file, RewriteRulesToBeAdded.txt(which is located in the GitOps repo)
Here are the steps in the Github action:
- name : Read serialized web item
shell: pwsh
run: |
$file = "./src/items/UrlRewritesWeb/UrlRewritesFile.yml"
Get-Content $file -Raw | Write-Host
- name: Get blob from web sitecore item
id: getbase64value
uses: mikefarah/yq@master
with:
cmd: yq '.SharedFields[] | select(.Hint == "Blob").Value' 'src/items/UrlRewritesWeb/UrlRewritesFile.yml'
- name: Display read-yaml output getbase64value
run: echo "${{ steps.getbase64value.outputs.result }}"
- name: Create rewriteRules file
id: rewriteRules
shell: pwsh
run: |
$rewriteRulesFileName = "RewriteRulesToBeAdded.txt"
Clear-Content ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\$rewriteRulesFileName"
$encodedBytes = [System.Convert]::FromBase64String("${{ steps.getbase64value.outputs.result }}")
Set-Content ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\$rewriteRulesFileName" -Value $encodedBytes -AsByteStream;
echo "::set-output name=REWRITE_RULES_FILE_NAME::${rewriteRulesFileName}"
- name : Read rewriteRules file
shell: pwsh
run: |
$file = ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\${{steps.rewriteRules.outputs.REWRITE_RULES_FILE_NAME}}"
Get-Content $file -Raw | Write-Host
Notice how we find the blob in the yml file, using yq – A lightweight and portable command-line for YAML. Wonderful gem 🙂
Prepare/format the rewrite rules (in the “temporary” txt file)
Great, now we have the rewrite rules(from Sitecore) in the temp file – RewriteRulesToBeAdded.txt. Next is to format the rewrite rules to merge into the manifest file.
Here are the steps in the Github action:
- name: Formatting the rewriteRules file
shell: pwsh
run: |
$sb = new-object -TypeName System.Text.StringBuilder
$6spaces = " " * 6
[System.IO.StreamReader]$sr = [System.IO.File]::Open("./k8/Sandbox/specs/${{github.event.client_payload.Ingressname}}/patches/${{steps.rewriteRules.outputs.REWRITE_RULES_FILE_NAME}}", [System.IO.FileMode]::Open)
while (-not $sr.EndOfStream){
$line = $sr.ReadLine()
if(![string]::IsNullOrWhiteSpace($line)) {
$sb.Append($6spaces)
$sb.Append($line.Trim())
$sb.AppendLine([String]::empty)
} else {
$sb.AppendLine([String]::empty)
}
}
$sr.Close()
Set-Content ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\${{steps.rewriteRules.outputs.REWRITE_RULES_FILE_NAME}}" -Value $sb.ToString()
- name : Read rewriteRules file
shell: pwsh
run: |
$file = ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\${{steps.rewriteRules.outputs.REWRITE_RULES_FILE_NAME}}"
Get-Content $file -Raw | Write-Host
Notice how the “6spaces” is (pre)added to each line.
The “read steps” are for debugging.
Merge/add the (formatted) rewrite rules to the manifest file – rewrite-rules-configuration.yaml
We will grab the content(rewrite rules) from the temp file, RewriteRulesToBeAdded.txt.
Then we will merge/put the content to the “editor’s section” in the manifest file – rewrite-rules-configuration.yaml.
Here are the steps in the Github action:
- name: Add rewrite rules to yaml file
shell: pwsh
run: |
$rewriteRulesContent = Get-Content ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\${{steps.rewriteRules.outputs.REWRITE_RULES_FILE_NAME}}" -raw
$sitecoreRewrites = "#############SitecoreRewrites###########"
$con = Get-Content ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\rewrite-rules-configuration.yaml" -raw
$toSave = $con.Substring(0, $con.IndexOf($sitecoreRewrites)+$sitecoreRewrites.Length)
$toSave = $toSave + "`r`n" + $rewriteRulesContent
Set-Content -Path ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\rewrite-rules-configuration.yaml" -Value $toSave
- name : Read updated yaml file
shell: pwsh
run: |
$file = ".\k8\Sandbox\specs\${{github.event.client_payload.Ingressname}}\patches\rewrite-rules-configuration.yaml"
Get-Content $file -Raw | Write-Host
Notice how we get values from previous steps – steps.rewriteRules.outputs.REWRITE_RULES_FILE_NAME
Change the file icon to “trafficlight_green”, by updating the serialized UrlRewritesFile item and using Sitecore Cli to make a “sitecore ser push”
It’s time to communicate with the editors and let them know it’s all good 🙂
First, we need to locate the file icon in the serialized (master) item file, UrlRewritesFile.yml.
Then replace the icon with the icon – “trafficlight_green”.
And finally, push the changes to Sitecore – sitecore ser push
The editors will be happy when they see the great(green) news 🙂
Here are the steps in the Github action:
- name: Get icon from sitecore master item
id: geticonvalue-before-success
uses: mikefarah/yq@master
with:
cmd: yq '.SharedFields[] | select(.Hint == "__Icon").Value' 'src/items/UrlRewritesMaster/UrlRewritesFile.yml'
- name : Set success value icon on sitecore master item
shell: pwsh
run: |
$iconSuccess="Office/32x32/trafficlight_green.png"
$geticonvalueToBeReplaced = "${{ steps.geticonvalue-before-success.outputs.result }}"
$con = Get-Content .\src\items\UrlRewritesMaster\UrlRewritesFile.yml
$con | % { $_.Replace($geticonvalueToBeReplaced,$iconSuccess) } | Set-Content .\src\items\UrlRewritesMaster\UrlRewritesFile.yml
- name : Read updated serialized sitecore master item
shell: pwsh
run: |
$file = "./src/items/UrlRewritesMaster/UrlRewritesFile.yml"
Get-Content $file -Raw | Write-Host
- name : Push content to cm
shell: bash
run: |
dotnet sitecore ser push --force -i Project.RewriteRules.Master
Notice that we are changing an item by updating the serialized file – UrlRewritesFile.yml. Not using any Sitecore item API. Instead, we just change the yml file and then do a “sitecore ser push” 🙂
Commit and push the changes to the GitOps repository
The last part is to commit and push the updated manifest file, rewrite-rules-configuration.yaml, to the GitOps repository.
Here are the steps in the Github action:
- name: Commit files
run: |
git config --local user.email github-actions@github.com
git config --local user.name github-actions
git commit -m "Updating rewrite rules from sitecore" -a
- name: Push changes
uses: ad-m/github-push-action@master
with:
github_token: ${{ secrets.REPO_PAT }}
branch: main
Lastly, Argo CD will sync the rewrite rules automatically to the Kubernetes cluster.
We now have the new rewrite rules running live on the site!
That’s all for now folks 😊
Visions In Code 
One thought on “Handle URL rewrites in a Kubernetes cluster running a Sitecore multisite solution with NGINX using GitOps with Sitecore CLI – Part 2”