Backup Sitecore content in GitHub Packages

Hello, Sitecorians. I hope you are well out there. 😊

I just went through the Sharpe’s Rifles (Sharpe) series, and what a show! Sean Bean does a great job starring as the grumpy soldier rising from the ranks during the napoleon wars. It’s an old show, but it’s a must-see. 😊

I will not spoil anything, but there is a particular scene that got to me. Sharpe is grumpy as always, and he can not understand why they should deliver this old artifact (“rag”) to the Spanish resistance.

Sharpe: Rag on a pole, rag on a pole! Do you really believe men will fight and die for a rag on a pole?
Hogan: You do, Richard. You do

https://tvtropes.org/pmwiki/pmwiki.php/Series/Sharpe

Ah, what a scene… Anyways let’s move on to today’s post 😊

I recently read a great post from Anton Tishchenko – Validate Your Sitecore Serialization Using Pre-commit Git Hook. I had to try it out, and it worked like a charm. It is a great idea to be sure that the serialized items are in order before a commit. Keep up the excellent work Anton Tishchenko!

Some boring naysayers are saying that pre-commit hooks are bad. I disagree! If the commit takes too long then you are serializing wrong! Never serialize content in your “developing repository”. If you need to have content serialized/saved, then keep it somewhere else. And that’s what today’s post will be about, to save/store content somewhere else in… GitHub Packages 😍

Last year I did a post about storing content in GitHub Packages:

Use Github Packages for storing Sitecore content as a nuget package, with the power of SITECORE CLI and GITHUB ACTIONS

There will be a slight update on the approach.

Instead of executing the GitHub Action within GitHub, we want to be able to do this from Sitecore. It means – Right click on a site and select “Backup Site” 

And instead of running the GitHub Actions locally (on the computer), we will have them running within our Kubernetes cluster using the wonderful actions-runner-controller (ARC). I will not go into how to set it up. If you are interested, you can check out my previous post – Run Sitecore CLI within your Kubernetes cluster using self-hosted GitHub actions.

Let’s start!

First up is to make a SPE script that will trigger the GitHub Action. The script will be a “Context Menu” script – Backup Site

The script will collect “needed” site info and trigger a GitHub Action. Let’s have a look at the script:

function Get-SiteInfo {

	Set-Location -Path .
	
	$currentItemId = Get-Item .
    $currentItemPath = $currentItemId.ItemPath 
	$medialibraryItem = Get-Item $currentItemPath"/Media"
	
	$virtualFolderField = [Sitecore.Data.Fields.MultilistField]$medialibraryItem.Fields["AdditionalChildren"]
	$firstInList = $virtualFolderField.GetItems() | Select-Object -first 1
	$mediaItem = Get-Item $firstInList.Id
 
	return @($currentItemId.Name, $currentItemId.Parent.Name, $currentItemPath, $mediaItem.ItemPath );

}

$GithubOrganization = $env:GithubOrganization 
$GithubRepo = $env:GithubRepo
$GithubToken = $env:GithubToken

$Uri = ('https://api.github.com/repos/{0}/{1}/dispatches' -f $GithubOrganization, $GithubRepo)


$siteName, $tenantName, $sitePath, $medialibraryPath  = Get-SiteInfo

$Obj = [PSCustomObject]@{
	event_type = "generate-content-package"
	client_payload = [PSCustomObject]@{ SiteTenant=$tenantName; SiteName=$SiteName; SitePath=$sitePath; SiteMedialibraryPath=$medialibraryPath; Env="TEST"}
}
$Body = $Obj | ConvertTo-JSON
Write-Host $Body
	
$params = @{
	ContentType = 'application/json'
	Headers     = @{
		'authorization' = "token $($GithubToken)"
		'accept'        = 'application/vnd.github.v3+json'
	}
	Method      = 'Post'
	URI         = $Uri
	Body        = $Body
}

Invoke-RestMethod @params -verbose
Write-Host "Triggering GitHub Action"

Write-Host "Check out the job you triggered at https://github.com/organization/my_repo/actions/workflows/GenerateContentPackage.yml"

The GitHub Action is listening to a “repository_dispatch” event. That’s why we are sending the event_type “generate-content-package”. Read more about it from my previous post – A powerful combination – Sitecore CLI, GitHub Actions, and Sitecore PowerShell Extensions 🙂

Ok, let’s move on to the GitHub Action

Here’s forty shillings on the drum
To those who volunteer to come,
To ‘list and fight the foe today
Over the Hills and far away.

Behave Rifleman Hagman, no singing!
Sorry for the interruption, let’s continue 🙂

We will create a “repository_dispatch” GitHub Action. It will listen to the event “generate-content-package”. But before we start, we need to add a folder containing nuget.exe and a nuspec file in the root(in our repository):

Let’s have a quick look at the nuspec file. It will work as a template since we need to copy and update it on the fly.

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/10/nuspec.xsd">
  <metadata>
    <id>BackupContent</id>
    <version>1.0.0</version>
    <authors>MeMyselfAndI</authors>
    <owners>MyOrganization</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Backup content Sitecore CLI package</description>
    <releaseNotes></releaseNotes>
    <copyright>2022</copyright>
    <tags></tags>
    <contentFiles>
      <files include="**\*.*" buildAction="Content" copyToOutput="true" />
    </contentFiles>
    <repository type="git" url="https://github.com/my_organization/my_repo" />
  </metadata>
  <files>
    <file src="*.*" target="Sitecore" />
  </files>
</package>

*without the repository element, we will not be able to push the NuGet package to GitHub Packages

Great, we are ready to set up the GitHub Action. The GitHub Action will do the following:

  • Check out the code
  • Do sitecore ser pull
  • Generate an .itempackage
  • Create a NuGet package containing the .itempackage
  • Push the NuGet package to the GitHub Packages

Here is the GitHub Action – GenerateContentPackage:

name: Dispatch event - Generate backup content package - Linux

on: 
   repository_dispatch:
    types: [generate-content-package]

      

jobs:
  generate-backup-content-package:
      name: Generate Backup Content Package  
      runs-on: [self-hosted, Linux]

      steps:
      - name: Checkout
        uses: actions/checkout@v3


      - name: Install dot net
        shell: bash
        run: | 
          wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
          sudo dpkg -i packages-microsoft-prod.deb
          rm packages-microsoft-prod.deb

          sudo apt-get update; \
          sudo apt-get install -y apt-transport-https && \
          sudo apt-get update && \
          sudo apt-get install -y dotnet-sdk-3.1
      

      - name : Install Mono
        shell: bash
        run: |
          sudo apt install -y gnupg ca-certificates
          sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
          echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
          sudo apt update
          sudo apt install -y mono-complete
          mono --version


      - name: Install sitecore cli
        shell: bash  
        run: | 
          dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
          dotnet tool restore
         
      - name: Login to sitecore
        shell: bash  
        run: |     
          dotnet sitecore login --client-credentials true --auth http://id --cm http://cm --allow-write true --client-id "My_Automation" --client-secret "PUT_YOUR_CLIENT_SECRET_HERE" --insecure


      - name : Create module.json file
        id: create-json-module
        uses: jsdaniell/create-json@1.1.2
        with:
          name: 'Content.module.json'
          dir: 'Sandbox.Sitecore/src/Project/${{ github.event.client_payload.SiteName }}/'
          json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"},{"name": "Dictionary","path": "${{ github.event.client_payload.SitePath }}/dictionary","database": "master"},{"name": "Presentation","path": "${{ github.event.client_payload.SitePath }}/presentation","database": "master"}, {"name": "Settings","path": "${{ github.event.client_payload.SitePath }}/settings","database": "master"}]}}'

      - name : Pull content from cm   
        shell: bash
        run: |
          dotnet sitecore ser pull -i Project.Content


      - name : Create Sitecore CLI package   
        shell: bash
        run: |
          dotnet sitecore ser package create -o BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }} -i Project.Content

      

      - name: Install PowerShell 7.2
        shell: bash  
        run: |     
          sudo apt-get update
          sudo apt-get install -y wget apt-transport-https software-properties-common
          wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
          sudo dpkg -i packages-microsoft-prod.deb
          sudo apt-get update
          sudo apt-get install -y powershell


      - name : Copy base nuspec file and update   
        shell: pwsh
        run: |
          Copy-Item -Path ".\BackupContent\nuget\BackupContent.nuspec" -Destination ".\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec"
          $con = Get-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec
          $con | % { $_.Replace("<id>BackupContent</id>", "<id>${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}</id>") } | % { $_.Replace("<version>1.0.0</version>", "<version>1.${{ github.run_number }}.0</version>") } | Set-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec


      
      - name: Create nuget package
        shell: pwsh  
        run: |     
          Invoke-Expression "mono ./BackupContent/nuget.exe pack BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec -Version 1.${{ github.run_number }}.0 -OutputDirectory BackupContent"         
          

      - name : Push nuget package to Github Packages   
        shell: pwsh
        run: |
          if (-not $(Get-PackageSource -Name "backupcontent" -ProviderName NuGet -ErrorAction Ignore))
          {
            mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/your_organization" -UserName ${{secrets.USER_NAME}} -Password ${{secrets.REPO_PAT}}
          }
          mono BackupContent/nuget.exe push  BackupContent/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.1.${{ github.run_number }}.0.nupkg -Source "backupcontent" -ApiKey ${{secrets.NUGET_PACKAGE_TOKEN}}                  
          
      

Let’s go through the GitHub Action step by step:

name: Dispatch event - Generate backup content package - Linux

on: 
   repository_dispatch:
    types: [generate-content-package]
      

Name of the action and that it’s listening to the event – generate-content-package

jobs:
  generate-backup-content-package:
      name: Generate Backup Content Package  
      runs-on: [self-hosted, Linux]
      

Name of the job and use the self-hosted(Linux) runner (GitHub action). Which runs within Kubernetes

      steps:
      - name: Checkout
        uses: actions/checkout@v3
      

Check out the code from the repo where the GitHub Action resides.

      - name: Install dot net
        shell: bash
        run: | 
          wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
          sudo dpkg -i packages-microsoft-prod.deb
          rm packages-microsoft-prod.deb

          sudo apt-get update; \
          sudo apt-get install -y apt-transport-https && \
          sudo apt-get update && \
          sudo apt-get install -y dotnet-sdk-3.1

Install Asp.Net Core 3.1

      - name : Install Mono
        shell: bash
        run: |
          sudo apt install -y gnupg ca-certificates
          sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
          echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
          sudo apt update
          sudo apt install -y mono-complete
          mono --version

Install Mono. We need it to be able to run executables – nuget.exe

      - name: Install sitecore cli
        shell: bash  
        run: | 
          dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
          dotnet tool restore
         
     

Install Sitecore Cli

      - name: Login to sitecore
        shell: bash  
        run: |     
          dotnet sitecore login --client-credentials true --auth http://id --cm http://cm --allow-write true --client-id "My_Automation" --client-secret "PUT_YOUR_CLIENT_SECRET_HERE" --insecure

      

Login to Sitecore. Because we are running the GitHub Action within the Kubernetes cluster, we can access cm and id by the service names. Notice we use HTTP, hence the –insecure.

      - name : Create module.json file
        id: create-json-module
        uses: jsdaniell/create-json@1.1.2
        with:
          name: 'Content.module.json'
          dir: 'Sandbox.Sitecore/src/Project/${{ github.event.client_payload.SiteName }}/'
          json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"},{"name": "Dictionary","path": "${{ github.event.client_payload.SitePath }}/dictionary","database": "master"},{"name": "Presentation","path": "${{ github.event.client_payload.SitePath }}/presentation","database": "master"}, {"name": "Settings","path": "${{ github.event.client_payload.SitePath }}/settings","database": "master"}]}}'

      

We create a module json file with the namespace Project.Content. Containing what we need to pull from CM. Notice how we use the parameters(payload) that weresent from Sitecore SPE:
{“name”:”Images”,”path”:”${{ github.event.client_payload.SiteMedialibraryPath }}”,”database”: “master”}

      - name : Pull content from cm   
        shell: bash
        run: |
          dotnet sitecore ser pull -i Project.Content


Here we pull the content from CM

      - name : Create Sitecore CLI package   
        shell: bash
        run: |
          dotnet sitecore ser package create -o BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }} -i Project.Content

      

Next is to create the .itempackage from the pulled content. Notice that we are giving the site name and what environment the site resides in(All from parameters)

      - name: Install PowerShell 7.2
        shell: bash  
        run: |     
          sudo apt-get update
          sudo apt-get install -y wget apt-transport-https software-properties-common
          wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
          sudo dpkg -i packages-microsoft-prod.deb
          sudo apt-get update
          sudo apt-get install -y powershell

      

Before we can do the last part, we need to install PowerShell.

      - name : Copy base nuspec file and update   
        shell: pwsh
        run: |
          Copy-Item -Path ".\BackupContent\nuget\BackupContent.nuspec" -Destination ".\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec"
          $con = Get-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec
          $con | % { $_.Replace("<id>BackupContent</id>", "<id>${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}</id>") } | % { $_.Replace("<version>1.0.0</version>", "<version>1.${{ github.run_number }}.0</version>") } | Set-Content .\BackupContent\nuget\${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec

  

We copy the template/base BackupContent.nuspec file, and give it a proper name => site name and what environment. Then we replace the id with the site name and environment, and set the version number.

      - name: Create nuget package
        shell: pwsh  
        run: |     
          Invoke-Expression "mono ./BackupContent/nuget.exe pack BackupContent/nuget/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.nuspec -Version 1.${{ github.run_number }}.0 -OutputDirectory BackupContent"         
          

It’s time to create the NuGet package. Notice how we use mono to execute “nuget.exe”.

      - name : Push nuget package to Github Packages   
        shell: pwsh
        run: |
          if (-not $(Get-PackageSource -Name "backupcontent" -ProviderName NuGet -ErrorAction Ignore))
          {
            mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/your_organization" -UserName ${{secrets.USER_NAME}} -Password ${{secrets.REPO_PAT}}
          }
          mono BackupContent/nuget.exe push  BackupContent/${{ github.event.client_payload.SiteName }}_${{ github.event.client_payload.Env }}.1.${{ github.run_number }}.0.nupkg -Source "backupcontent" -ApiKey ${{secrets.NUGET_PACKAGE_TOKEN}}                  
          
      

And finally, we push the NuGet package to GitHub Packages

Let’s have a look at the GitHub Packages. There it is 🙂

How about we download the NuGet package and have a look at it:

Great and let’s see if we have a .itempackage. Let’s have a look in the Sitecore folder:

Success!

Stay tuned for part 2, where we’ll install the NuGet package into Sitecore.

That’s all for now folks 😊


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.