Hello, Sitecorians. I hope you are well out there. 😊
I just went through the Sharpe’s Rifles (Sharpe) series, and what a show! Sean Bean does a great job starring as the grumpy soldier rising from the ranks during the napoleon wars. It’s an old show, but it’s a must-see. 😊
I will not spoil anything, but there is a particular scene that got to me. Sharpe is grumpy as always, and he can not understand why they should deliver this old artifact (“rag”) to the Spanish resistance.
Sharpe: Rag on a pole, rag on a pole! Do you really believe men will fight and die for a rag on a pole?
https://tvtropes.org/pmwiki/pmwiki.php/Series/Sharpe
Hogan: You do, Richard. You do
Ah, what a scene… Anyways let’s move on to today’s post 😊
I recently read a great post from Anton Tishchenko – Validate Your Sitecore Serialization Using Pre-commit Git Hook. I had to try it out, and it worked like a charm. It is a great idea to be sure that the serialized items are in order before a commit. Keep up the excellent work Anton Tishchenko!
Some boring naysayers are saying that pre-commit hooks are bad. I disagree! If the commit takes too long then you are serializing wrong! Never serialize content in your “developing repository”. If you need to have content serialized/saved, then keep it somewhere else. And that’s what today’s post will be about, to save/store content somewhere else in… GitHub Packages 😍
Last year I did a post about storing content in GitHub Packages:
There will be a slight update on the approach.
Instead of executing the GitHub Action within GitHub, we want to be able to do this from Sitecore. It means – Right click on a site and select “Backup Site”
And instead of running the GitHub Actions locally (on the computer), we will have them running within our Kubernetes cluster using the wonderful actions-runner-controller (ARC). I will not go into how to set it up. If you are interested, you can check out my previous post – Run Sitecore CLI within your Kubernetes cluster using self-hosted GitHub actions.
Let’s start!
First up is to make a SPE script that will trigger the GitHub Action. The script will be a “Context Menu” script – Backup Site
The script will collect “needed” site info and trigger a GitHub Action. Let’s have a look at the script:
function Get-SiteInfo {
Set-Location -Path .
$currentItemId = Get-Item .
$currentItemPath = $currentItemId.ItemPath
$medialibraryItem = Get-Item $currentItemPath"/Media"
$virtualFolderField = [Sitecore.Data.Fields.MultilistField]$medialibraryItem.Fields["AdditionalChildren"]
$firstInList = $virtualFolderField.GetItems() | Select-Object -first 1
$mediaItem = Get-Item $firstInList.Id
return @($currentItemId.Name, $currentItemId.Parent.Name, $currentItemPath, $mediaItem.ItemPath );
}
$GithubOrganization = $env:GithubOrganization
$GithubRepo = $env:GithubRepo
$GithubToken = $env:GithubToken
$Uri = ('https://api.github.com/repos/{0}/{1}/dispatches' -f $GithubOrganization, $GithubRepo)
$siteName, $tenantName, $sitePath, $medialibraryPath = Get-SiteInfo
$Obj = [PSCustomObject]@{
event_type = "generate-content-package"
client_payload = [PSCustomObject]@{ SiteTenant=$tenantName; SiteName=$SiteName; SitePath=$sitePath; SiteMedialibraryPath=$medialibraryPath; Env="TEST"}
}
$Body = $Obj | ConvertTo-JSON
Write-Host $Body
$params = @{
ContentType = 'application/json'
Headers = @{
'authorization' = "token $($GithubToken)"
'accept' = 'application/vnd.github.v3+json'
}
Method = 'Post'
URI = $Uri
Body = $Body
}
Invoke-RestMethod @params -verbose
Write-Host "Triggering GitHub Action"
Write-Host "Check out the job you triggered at https://github.com/organization/my_repo/actions/workflows/GenerateContentPackage.yml"
The GitHub Action is listening to a “repository_dispatch” event. That’s why we are sending the event_type “generate-content-package”. Read more about it from my previous post – A powerful combination – Sitecore CLI, GitHub Actions, and Sitecore PowerShell Extensions 🙂
Ok, let’s move on to the GitHub Action
Here’s forty shillings on the drum
To those who volunteer to come,
To ‘list and fight the foe today
Over the Hills and far away.
Behave Rifleman Hagman, no singing!
Sorry for the interruption, let’s continue 🙂
We will create a “repository_dispatch” GitHub Action. It will listen to the event “generate-content-package”. But before we start, we need to add a folder containing nuget.exe and a nuspec file in the root(in our repository):
Let’s have a quick look at the nuspec file. It will work as a template since we need to copy and update it on the fly.
<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/10/nuspec.xsd">
<metadata>
<id>BackupContent</id>
<version>1.0.0</version>
<authors>MeMyselfAndI</authors>
<owners>MyOrganization</owners>
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<description>Backup content Sitecore CLI package</description>
<releaseNotes></releaseNotes>
<copyright>2022</copyright>
<tags></tags>
<contentFiles>
<files include="**\*.*" buildAction="Content" copyToOutput="true" />
</contentFiles>
<repository type="git" url="https://github.com/my_organization/my_repo" />
</metadata>
<files>
<file src="*.*" target="Sitecore" />
</files>
</package>
*without the repository element, we will not be able to push the NuGet package to GitHub Packages
Great, we are ready to set up the GitHub Action. The GitHub Action will do the following:
- Check out the code
- Do sitecore ser pull
- Generate an .itempackage
- Create a NuGet package containing the .itempackage
- Push the NuGet package to the GitHub Packages
Here is the GitHub Action – GenerateContentPackage:
name: Dispatch event - Generate backup content package - Linux
on:
repository_dispatch:
types: [generate-content-package]
jobs:
generate-backup-content-package:
name: Generate Backup Content Package
runs-on: [self-hosted, Linux]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install dot net
shell: bash
run: |
wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1
- name : Install Mono
shell: bash
run: |
sudo apt install -y gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
sudo apt update
sudo apt install -y mono-complete
mono --version
- name: Install sitecore cli
shell: bash
run: |
dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
dotnet tool restore
- name: Login to sitecore
shell: bash
run: |
dotnet sitecore login --client-credentials true --auth http://id --cm http://cm --allow-write true --client-id "My_Automation" --client-secret "PUT_YOUR_CLIENT_SECRET_HERE" --insecure
- name : Create module.json file
id: create-json-module
uses: jsdaniell/create-json@1.1.2
with:
name: 'Content.module.json'
dir: 'Sandbox.Sitecore/src/Project/${{ github.event.client_payload.SiteName }}/'
json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"},{"name": "Dictionary","path": "${{ github.event.client_payload.SitePath }}/dictionary","database": "master"},{"name": "Presentation","path": "${{ github.event.client_payload.SitePath }}/presentation","database": "master"}, {"name": "Settings","path": "${{ github.event.client_payload.SitePath }}/settings","database": "master"}]}}'
- name : Pull content from cm
shell: bash
run: |
dotnet sitecore ser pull -i Project.Content
- name : Create Sitecore CLI package
shell: bash
run: |
dotnet sitecore ser package create -o BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }} -i Project.Content
- name: Install PowerShell 7.2
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y wget apt-transport-https software-properties-common
wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install -y powershell
- name : Copy base nuspec file and update
shell: pwsh
run: |
Copy-Item -Path ".\BackupContent\nuget\BackupContent.nuspec" -Destination ".\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec"
$con = Get-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec
$con | % { $_.Replace("<id>BackupContent</id>", "<id>${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}</id>") } | % { $_.Replace("<version>1.0.0</version>", "<version>1.${{ github.run_number }}.0</version>") } | Set-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec
- name: Create nuget package
shell: pwsh
run: |
Invoke-Expression "mono ./BackupContent/nuget.exe pack BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec -Version 1.${{ github.run_number }}.0 -OutputDirectory BackupContent"
- name : Push nuget package to Github Packages
shell: pwsh
run: |
if (-not $(Get-PackageSource -Name "backupcontent" -ProviderName NuGet -ErrorAction Ignore))
{
mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/your_organization" -UserName ${{secrets.USER_NAME}} -Password ${{secrets.REPO_PAT}}
}
mono BackupContent/nuget.exe push BackupContent/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.1.${{ github.run_number }}.0.nupkg -Source "backupcontent" -ApiKey ${{secrets.NUGET_PACKAGE_TOKEN}}
Let’s go through the GitHub Action step by step:
name: Dispatch event - Generate backup content package - Linux
on:
repository_dispatch:
types: [generate-content-package]
Name of the action and that it’s listening to the event – generate-content-package
jobs:
generate-backup-content-package:
name: Generate Backup Content Package
runs-on: [self-hosted, Linux]
Name of the job and use the self-hosted(Linux) runner (GitHub action). Which runs within Kubernetes
steps:
- name: Checkout
uses: actions/checkout@v3
Check out the code from the repo where the GitHub Action resides.
- name: Install dot net
shell: bash
run: |
wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1
Install Asp.Net Core 3.1
- name : Install Mono
shell: bash
run: |
sudo apt install -y gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
sudo apt update
sudo apt install -y mono-complete
mono --version
Install Mono. We need it to be able to run executables – nuget.exe
- name: Install sitecore cli
shell: bash
run: |
dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
dotnet tool restore
Install Sitecore Cli
- name: Login to sitecore
shell: bash
run: |
dotnet sitecore login --client-credentials true --auth http://id --cm http://cm --allow-write true --client-id "My_Automation" --client-secret "PUT_YOUR_CLIENT_SECRET_HERE" --insecure
Login to Sitecore. Because we are running the GitHub Action within the Kubernetes cluster, we can access cm and id by the service names. Notice we use HTTP, hence the –insecure.
- name : Create module.json file
id: create-json-module
uses: jsdaniell/create-json@1.1.2
with:
name: 'Content.module.json'
dir: 'Sandbox.Sitecore/src/Project/${{ github.event.client_payload.SiteName }}/'
json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"},{"name": "Dictionary","path": "${{ github.event.client_payload.SitePath }}/dictionary","database": "master"},{"name": "Presentation","path": "${{ github.event.client_payload.SitePath }}/presentation","database": "master"}, {"name": "Settings","path": "${{ github.event.client_payload.SitePath }}/settings","database": "master"}]}}'
We create a module json file with the namespace Project.Content. Containing what we need to pull from CM. Notice how we use the parameters(payload) that weresent from Sitecore SPE:
{“name”:”Images”,”path”:”${{ github.event.client_payload.SiteMedialibraryPath }}”,”database”: “master”}
- name : Pull content from cm
shell: bash
run: |
dotnet sitecore ser pull -i Project.Content
Here we pull the content from CM
- name : Create Sitecore CLI package
shell: bash
run: |
dotnet sitecore ser package create -o BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }} -i Project.Content
Next is to create the .itempackage from the pulled content. Notice that we are giving the site name and what environment the site resides in(All from parameters)
- name: Install PowerShell 7.2
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y wget apt-transport-https software-properties-common
wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install -y powershell
Before we can do the last part, we need to install PowerShell.
- name : Copy base nuspec file and update
shell: pwsh
run: |
Copy-Item -Path ".\BackupContent\nuget\BackupContent.nuspec" -Destination ".\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec"
$con = Get-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec
$con | % { $_.Replace("<id>BackupContent</id>", "<id>${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}</id>") } | % { $_.Replace("<version>1.0.0</version>", "<version>1.${{ github.run_number }}.0</version>") } | Set-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec
We copy the template/base BackupContent.nuspec file, and give it a proper name => site name and what environment. Then we replace the id with the site name and environment, and set the version number.
- name: Create nuget package
shell: pwsh
run: |
Invoke-Expression "mono ./BackupContent/nuget.exe pack BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec -Version 1.${{ github.run_number }}.0 -OutputDirectory BackupContent"
It’s time to create the NuGet package. Notice how we use mono to execute “nuget.exe”.
- name : Push nuget package to Github Packages
shell: pwsh
run: |
if (-not $(Get-PackageSource -Name "backupcontent" -ProviderName NuGet -ErrorAction Ignore))
{
mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/your_organization" -UserName ${{secrets.USER_NAME}} -Password ${{secrets.REPO_PAT}}
}
mono BackupContent/nuget.exe push BackupContent/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.1.${{ github.run_number }}.0.nupkg -Source "backupcontent" -ApiKey ${{secrets.NUGET_PACKAGE_TOKEN}}
And finally, we push the NuGet package to GitHub Packages
Let’s have a look at the GitHub Packages. There it is 🙂
How about we download the NuGet package and have a look at it:
Great and let’s see if we have a .itempackage. Let’s have a look in the Sitecore folder:
Success!
Stay tuned for part 2, where we’ll install the NuGet package into Sitecore.
That’s all for now folks 😊
Visions In Code 