Happy happy days, dear Sitecore developers. It’s spring in the air, and the sun is giving us new energy to do some serious sitecoring 🙂
Last year I wrote a post about storing content in GitHub Packages – Backup Sitecore content in GitHub Packages. And I promised to do a follow-up on how to install/restore GitHub Packages.
Today’s post will be about restoring(installing) GitHub(NuGet) Packages into your Sitecore environment. This post will be an update to one of my older posts – Use Github Packages for storing Sitecore content as a NuGet package, with the power of SITECORE CLI and GITHUB ACTIONS
Let’s begin 🙂
The idea is this:
- When right-clicking on a website tenant from the Sitecore Content Editor, we want to restore/install a Sitecore content (NuGet)package from one of our environments(test, stage, or prod).
*And yes, we could work with the good old Sitecore packages, but it’s not fun 😉 - We also want to select the environment from which we want to restore/install the package. For example, let’s say we want to install a Sitecore content (NuGet) package from production to our local Sitecore environment.
- Finally, it’s essential that we only work with cool techniques… Like Sitecore Cli, Sitecore PowerShell Extensions, GitHub Packages, and GitHub Actions 😎
The first sections(1 and 2) will use SPE(Sitecore PowerShell Extensions). Again I must give a massive shout-out to the Sitecore PowerShell Extensions people at Sitecore and Michael West. If you want to support Michael in his great work, I suggest you sponsor him at https://github.com/sponsors/michaellwest
We will also use localtunnel for exposing our local URLs (id and cm). And here I want to give a shout-out to Martin Miles(aka SitecoreMartin) for his excellent blog post – Tunneling out Sitecore 10.3 from a local machine containers for the full global access
I was thinking of using Ngrok, but thanks to Martin, there is now a better option – localtunnel (And it’s free 😉)
*The only caveat is that it does not support VPN
We will make a script that lists the available GitHub (NuGet) packages for a specific website. The script will then trigger a GitHubAction that will install a NuGet package and push Sitecore content to an environment(In this scenario, localhost)
Here is the PowerShell script(which is added to the Context Menu):
function Get-SiteInfo {
Set-Location -Path .
$currentItemId = Get-Item .
$currentItemPath = $currentItemId.ItemPath
$medialibraryItem = Get-Item $currentItemPath"/Media"
$virtualFolderField = [Sitecore.Data.Fields.MultilistField]$medialibraryItem.Fields["AdditionalChildren"]
$firstInList = $virtualFolderField.GetItems() | Select-Object -first 1
$mediaItem = Get-Item $firstInList.Id
return @($currentItemId.Name, $currentItemId.Parent.Name, $currentItemPath, $mediaItem.ItemPath );
}
function Get-Packages-BySiteName {
Param($SiteName)
$Headers=@{"Accept"="application/vnd.github+json"
"Authorization"="Bearer $GithubToken"
"X-GitHub-Api-Version"="2022-11-28"
}
$Uri = ('https://api.github.com/orgs/{0}/packages?package_type=nuget' -f $GithubOrganization)
$options = [Ordered]@{}
Try
{
$result = Invoke-WebRequest -UseBasicParsing -H $Headers $Uri | ConvertFrom-Json
$result.Where{$_.Name -Match $SiteName}.Name | ForEach-Object -Process {$options +=@{$_=$_}}
}
Catch
{
}
return $options
}
$GithubOrganization = $env:GithubOrg
$GithubRepo = $env:GithubRepo
$GithubToken = $env:GithubToken
$ClientSecret = $env:$ClientSecret
$siteName, $tenantName, $sitePath, $medialibraryPath = Get-SiteInfo
$nugetPackages=Get-Packages-BySiteName $siteName
$localHostInstructions=@"
1. Install localtunnel:
-------------------
npm install -g localtunnel
-------------------
2. Open a powershell window run the following for your id host:
-------------------
lt --local-host id.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://blabla-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button
Copy and paste the url to "Your exposed id host"
3. Open a second powershell window to run the following command for your cm host:
-------------------
lt --local-host cm.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://xyasas-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button
Copy and paste the url to "Your exposed cm host"
"@
$inputProps = @{
Parameters=@(
@{Name="LocalHostInstructions"; Title="Install localtunnel";Value=$localHostInstructions;Lines=25}
@{Name="IdHost"; Title="Your exposed id host";Mandatory = $true}
@{Name="CmHost"; Title="Your exposed cm host";Mandatory = $true}
@{Name="NugetPackageToRestore"; Title="Select a package to restore from"; Options=$nugetPackages}
)
Validator = {
if (!$variables.SitecoreIdHost.Value.Contains("https://")) {
$variables.SitecoreIdHost.Error = "https is missing"
}
if (!$variables.SitecoreCmHost.Value.Contains("https://")) {
$variables.SitecoreCmHost.Error = "https is missing"
}
if (!$variables.NugetPackageToRestore.Value) {
$variables.NugetPackageToRestore.Error = "Site has no backup packages"
}
}
Title = "Restore site backup"
Description = "Restore/install sitecore content to selected website."
Width = 800
Height = 300
ShowHints = $true
OkButtonName = "VALIDATE & RESTORE"
}
$result = Read-Variable @inputProps
if($result -eq "cancel"){
Write-Host "Not triggering GitHub Action"
Close-Window
}
if($result -ne "cancel"){
$Uri = ('https://api.github.com/repos/{0}/{1}/dispatches' -f $GithubOrganization, $GithubRepo)
$Obj = [PSCustomObject]@{
event_type = "deploy-content-package"
client_payload = [PSCustomObject]@{ IdHost=$IdHost; CmHost=$CmHost; ClientSecret=$ClientSecret; SiteTenant=$tenantName; SiteName=$siteName; SitePath=$sitePath; SiteMedialibraryPath=$medialibraryPath; NugetPackageToRestore=$nugetPackageToRestore}
}
$Body = $Obj | ConvertTo-JSON
Write-Host $Body
$params = @{
ContentType = 'application/json'
Headers = @{
'authorization' = "token $($GithubToken)"
'accept' = 'application/vnd.github.v3+json'
}
Method = 'Post'
URI = $Uri
Body = $Body
}
Invoke-RestMethod @params -verbose
Write-Host "Triggering GitHub Action"
Write-Host "Check out the job you triggered at https://github.com/sandbox/therepo/actions/workflows/DeployContentPackage.yml"
exit
}
Close-Window
Let me break down the script for you 🙂
First, we set some secret stuff. This is for GitHub Packages and triggering GitHubActions. They are all environment variables that are set in the docker-compose file.
$GithubOrganization = $env:GithubOrg
$GithubRepo = $env:GithubRepo
$GithubToken = $env:GithubToken
$ClientSecret = $env:$ClientSecret
We need to find out what website we want to restore/install content to and some other information:
$siteName, $tenantName, $sitePath, $medialibraryPath = Get-SiteInfo
function Get-SiteInfo {
Set-Location -Path .
$currentItemId = Get-Item .
$currentItemPath = $currentItemId.ItemPath
$medialibraryItem = Get-Item $currentItemPath"/Media"
$virtualFolderField = [Sitecore.Data.Fields.MultilistField]$medialibraryItem.Fields["AdditionalChildren"]
$firstInList = $virtualFolderField.GetItems() | Select-Object -first 1
$mediaItem = Get-Item $firstInList.Id
return @($currentItemId.Name, $currentItemId.Parent.Name, $currentItemPath, $mediaItem.ItemPath );
}
Now to some cool stuff, I wanted to get available GitHub Packages (NuGet) for a specific site and return the result as dropdown options.
*We are storing Sitecore Cli itempackages in NuGet packages(at GitHub Packages)
To get info from GitHub Packages, there is this excellent rest api that allows you to query GitHub Packages.
In our scenario, we fetch the packages info, parse the result to JSON, and filter out available NuGet packages for a specific site name.
$nugetPackages=Get-Packages-BySiteName $siteName
function Get-Packages-BySiteName {
Param($SiteName)
$Headers=@{"Accept"="application/vnd.github+json"
"Authorization"="Bearer $GithubToken"
"X-GitHub-Api-Version"="2022-11-28"
}
$Uri = ('https://api.github.com/orgs/{0}/packages?package_type=nuget' -f $GithubOrganization)
$options = [Ordered]@{}
Try
{
$result = Invoke-WebRequest -UseBasicParsing -H $Headers $Uri | ConvertFrom-Json
$result.Where{$_.Name -Match $SiteName}.Name | ForEach-Object -Process {$options +=@{$_=$_}}
}
Catch
{
}
return $options
}
We also give some proper instructions on making cm and id publicly available by installing localtunnel.
$localHostInstructions=@"
1. Install localtunnel:
-------------------
npm install -g localtunnel
-------------------
2. Open a powershell window run the following for your id host:
-------------------
lt --local-host id.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://blabla-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button
Copy and paste the url to "Your exposed id host"
3. Open a second powershell window to run the following command for your cm host:
-------------------
lt --local-host cm.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://xyasas-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button
Copy and paste the url to "Your exposed cm host"
"@
When clicking the “Ok button”(VALIDATE & RESTORE), we will collect info from the form and trigger a GitHubAction with event_type “deploy-content-package”. This means we will have a GitHub Action that listens to the event “deploy-content-package”. And keep in mind that we have made CM and ID available publicly. This means that the GitHubAction will have access to our local environment 😉
if($result -ne "cancel"){
$Uri = ('https://api.github.com/repos/{0}/{1}/dispatches' -f $GithubOrganization, $GithubRepo)
$Obj = [PSCustomObject]@{
event_type = "deploy-content-package"
client_payload = [PSCustomObject]@{ IdHost=$IdHost; CmHost=$CmHost; ClientSecret=$ClientSecret; SiteTenant=$tenantName; SiteName=$siteName; SitePath=$sitePath; SiteMedialibraryPath=$medialibraryPath; NugetPackageToRestore=$nugetPackageToRestore}
}
$Body = $Obj | ConvertTo-JSON
Write-Host $Body
$params = @{
ContentType = 'application/json'
Headers = @{
'authorization' = "token $($GithubToken)"
'accept' = 'application/vnd.github.v3+json'
}
Method = 'Post'
URI = $Uri
Body = $Body
}
Invoke-RestMethod @params -verbose
Write-Host "Triggering GitHub Action"
Write-Host "Check out the job you triggered at https://github.com/sandbox/therepo/actions/workflows/DeployContentPackage.yml"
exit
}
And here is what the dialog form looks like:
The final part is the GitHub Action, which will listen to the event “deploy-content-package”. Here we have the GitHub Action:
name: Dispatch event - Deploy backup content package to local cm - Linux
run-name: Dispatch event - Deploy backup package ${{ github.event.client_payload.NugetPackageToRestore }} to local host
on:
repository_dispatch:
types: [deploy-content-package]
jobs:
install-nuget:
name: Install NuGet package
runs-on: self-hosted
steps:
- name: Checkout SandboxSolution repo
uses: actions/checkout@v3
with:
repository: sandbox/SandboxSolution
token: ${{ secrets.REPO_PAT }}
ref: test
- name: Install dot net
shell: bash
run: |
wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1
- name : Install Mono
shell: bash
run: |
sudo apt install -y gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
sudo apt update
sudo apt install -y mono-complete
mono --version
- name: Install PowerShell 7.2
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y wget apt-transport-https software-properties-common
wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install -y powershell
- name: Install sitecore cli
shell: bash
run: |
dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
dotnet tool restore
- name : Create module.json file
id: create-json-module
uses: jsdaniell/create-json@1.1.2
with:
name: 'Content.module.json'
dir: 'SandboxSolution/src/Project/${{ github.event.client_payload.SiteName }}/'
json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"}]}}'
- name : Install NuGet package
shell: pwsh
run: |
mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/sandbox/index.json" -UserName ${{secrets.USERNAME}} -Password ${{secrets.NUGET_PACKAGE_TOKEN}}
mono BackupContent/nuget.exe install ${{ github.event.client_payload.NugetPackageToRestore }} -Source "backupcontent" -OutputDirectory BackupContent -x
- name : Deploy SCS package to CM
shell: pwsh
run: |
# Login to ID Server
dotnet sitecore login --client-credentials true --auth ${{github.event.client_payload.IdHost}} --cm ${{github.event.client_payload.CmHost}} --allow-write true --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}"
# Install package
dotnet sitecore ser package install --package BackupContent/${{ github.event.client_payload.NugetPackageToRestore }}/Sitecore/${{ github.event.client_payload.NugetPackageToRestore }}.itempackage --cm ${{github.event.client_payload.CmHost}} --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}" -i Project.Content
Let me break it down for you 🙂
We set the event that will trigger the GitHub Action. Notice that we are using a self-hosted GitHub Action. This is not necessary. You can use a “public” GitHub Action.
However, we have the GitHub Actions hosted in our Kubernetes cluster, and that is how you should do it(If you have a running Kubernetes cluster). Check out my previous post on how to set up Actions Runner Controller (ARC) in your Kubernetes cluster – Run Sitecore CLI within your Kubernetes cluster using self-hosted GitHub actions
name: Dispatch event - Deploy backup content package to local cm - Linux
run-name: Dispatch event - Deploy backup package ${{ github.event.client_payload.NugetPackageToRestore }} to local host
on:
repository_dispatch:
types: [deploy-content-package]
jobs:
install-nuget:
name: Install NuGet package
runs-on: self-hosted
Since we are running on a “Linux” GitHub Action, we must install Dot Net, Mono, and PowerShell. Dot Net and Powershell need no explanation. Mono is required if you want to execute an “executable” like Nuget.exe and 🙂
And, of course, install Sitecore Cli, the best command tool ever!
- name: Install dot net
shell: bash
run: |
wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1
- name : Install Mono
shell: bash
run: |
sudo apt install -y gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
sudo apt update
sudo apt install -y mono-complete
mono --version
- name: Install PowerShell 7.2
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y wget apt-transport-https software-properties-common
wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install -y powershell
- name: Install sitecore cli
shell: bash
run: |
dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
dotnet tool restore
Because we have itempackages in our NuGet Packages, we need to set up/create a module.json file. Telling what content should be pushed to our local Sitecore environment. Notice how we use github.event.client_payload, which is set( and sent) from the Sitecore SPE script 🙂
- name : Create module.json file
id: create-json-module
uses: jsdaniell/create-json@1.1.2
with:
name: 'Content.module.json'
dir: 'SandboxSolution/src/Project/${{ github.event.client_payload.SiteName }}/'
json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"}]}}'
It’s time to install the “requested” NuGet Package – github.event.client_payload.NugetPackageToRestore. Notice how we use mono to execute nuget.exe
- name : Install NuGet package
shell: pwsh
run: |
mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/sandbox/index.json" -UserName ${{secrets.USERNAME}} -Password ${{secrets.NUGET_PACKAGE_TOKEN}}
mono BackupContent/nuget.exe install ${{ github.event.client_payload.NugetPackageToRestore }} -Source "backupcontent" -OutputDirectory BackupContent -x
In the final part, the workflow will log in and push the Sitecore content(itempackage) to my local Sitecore environment. And since we have made CM and ID publicly available, it will work like a charm 🙂
- name : Deploy SCS package to CM
shell: pwsh
run: |
# Login to ID Server
dotnet sitecore login --client-credentials true --auth ${{github.event.client_payload.IdHost}} --cm ${{github.event.client_payload.CmHost}} --allow-write true --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}"
# Install package
dotnet sitecore ser package install --package BackupContent/${{ github.event.client_payload.NugetPackageToRestore }}/Sitecore/${{ github.event.client_payload.NugetPackageToRestore }}.itempackage --cm ${{github.event.client_payload.CmHost}} --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}" -i Project.Content
SPE, Sitecore Cli, GitHub Actions, and GitHub Packages truly rock!
That’s all for now folks 😊
Visions In Code 