Install GitHub(NuGet) Packages to a Sitecore environment using Sitecore Cli, SPE and GitHub Actions

Posted on by gorhal

Happy happy days, dear Sitecore developers. It’s spring in the air, and the sun is giving us new energy to do some serious sitecoring 🙂

Last year I wrote a post about storing content in GitHub Packages – Backup Sitecore content in GitHub Packages. And I promised to do a follow-up on how to install/restore GitHub Packages.

Today’s post will be about restoring(installing) GitHub(NuGet) Packages into your Sitecore environment. This post will be an update to one of my older posts – Use Github Packages for storing Sitecore content as a NuGet package, with the power of SITECORE CLI and GITHUB ACTIONS

Let’s begin 🙂

The idea is this:

  1. When right-clicking on a website tenant from the Sitecore Content Editor, we want to restore/install a Sitecore content (NuGet)package from one of our environments(test, stage, or prod).
    *And yes, we could work with the good old Sitecore packages, but it’s not fun 😉
  2. We also want to select the environment from which we want to restore/install the package. For example, let’s say we want to install a Sitecore content (NuGet) package from production to our local Sitecore environment.
  3. Finally, it’s essential that we only work with cool techniques… Like Sitecore Cli, Sitecore PowerShell Extensions, GitHub Packages, and GitHub Actions 😎

The first sections(1 and 2) will use SPE(Sitecore PowerShell Extensions). Again I must give a massive shout-out to the Sitecore PowerShell Extensions people at Sitecore and Michael West. If you want to support Michael in his great work, I suggest you sponsor him at https://github.com/sponsors/michaellwest

We will also use localtunnel for exposing our local URLs (id and cm). And here I want to give a shout-out to Martin Miles(aka SitecoreMartin) for his excellent blog post – Tunneling out Sitecore 10.3 from a local machine containers for the full global access

I was thinking of using Ngrok, but thanks to Martin, there is now a better option – localtunnel (And it’s free 😉)
*The only caveat is that it does not support VPN

We will make a script that lists the available GitHub (NuGet) packages for a specific website. The script will then trigger a GitHubAction that will install a NuGet package and push Sitecore content to an environment(In this scenario, localhost)

Here is the PowerShell script(which is added to the Context Menu):

function Get-SiteInfo {

	Set-Location -Path .
	
	$currentItemId = Get-Item .
    $currentItemPath = $currentItemId.ItemPath 
	$medialibraryItem = Get-Item $currentItemPath"/Media"
	
	$virtualFolderField = [Sitecore.Data.Fields.MultilistField]$medialibraryItem.Fields["AdditionalChildren"]
	$firstInList = $virtualFolderField.GetItems() | Select-Object -first 1
	$mediaItem = Get-Item $firstInList.Id
 
 
	return @($currentItemId.Name, $currentItemId.Parent.Name, $currentItemPath, $mediaItem.ItemPath );

}

function Get-Packages-BySiteName {

    Param($SiteName)

    $Headers=@{"Accept"="application/vnd.github+json"
    "Authorization"="Bearer $GithubToken"
    "X-GitHub-Api-Version"="2022-11-28"
    }

    $Uri = ('https://api.github.com/orgs/{0}/packages?package_type=nuget' -f $GithubOrganization)

    $options = [Ordered]@{}
    
    Try
    {
       $result = Invoke-WebRequest -UseBasicParsing -H $Headers $Uri |  ConvertFrom-Json
       $result.Where{$_.Name -Match $SiteName}.Name | ForEach-Object -Process {$options +=@{$_=$_}}
    }
    Catch
    {
       
    }

    return $options

}

$GithubOrganization = $env:GithubOrg

$GithubRepo = $env:GithubRepo

$GithubToken = $env:GithubToken

$ClientSecret = $env:$ClientSecret

$siteName, $tenantName, $sitePath, $medialibraryPath  = Get-SiteInfo

$nugetPackages=Get-Packages-BySiteName $siteName

$localHostInstructions=@"
1. Install localtunnel:
-------------------
npm install -g localtunnel
-------------------

2. Open a powershell window run the following for your id host:
-------------------
lt --local-host id.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://blabla-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button

Copy and paste the url to "Your exposed id host"

3. Open a second powershell window to run the following command for your cm host:
-------------------
lt --local-host cm.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://xyasas-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button

Copy and paste the url to "Your exposed cm host"

"@

$inputProps = @{
    Parameters=@(
        @{Name="LocalHostInstructions"; Title="Install localtunnel";Value=$localHostInstructions;Lines=25}
        @{Name="IdHost"; Title="Your exposed id host";Mandatory = $true}
        @{Name="CmHost"; Title="Your exposed cm host";Mandatory = $true}
        @{Name="NugetPackageToRestore"; Title="Select a package to restore from"; Options=$nugetPackages}
    )
    Validator = {
        
        if (!$variables.SitecoreIdHost.Value.Contains("https://")) {
            $variables.SitecoreIdHost.Error = "https is missing"
        } 
         
        if (!$variables.SitecoreCmHost.Value.Contains("https://")) {
            $variables.SitecoreCmHost.Error = "https is missing"
        } 
        
        if (!$variables.NugetPackageToRestore.Value) {
            $variables.NugetPackageToRestore.Error = "Site has no backup packages"
        } 

    }
    Title = "Restore site backup"
    Description = "Restore/install sitecore content to selected website."
    Width = 800
    Height = 300
    ShowHints = $true
    OkButtonName = "VALIDATE & RESTORE"
   
}



$result = Read-Variable @inputProps
if($result -eq "cancel"){
    Write-Host "Not triggering GitHub Action"
    Close-Window
}

if($result -ne "cancel"){

        $Uri = ('https://api.github.com/repos/{0}/{1}/dispatches' -f $GithubOrganization, $GithubRepo)

        
        
        $Obj = [PSCustomObject]@{
        	event_type = "deploy-content-package"
        	client_payload = [PSCustomObject]@{ IdHost=$IdHost; CmHost=$CmHost; ClientSecret=$ClientSecret; SiteTenant=$tenantName; SiteName=$siteName; SitePath=$sitePath; SiteMedialibraryPath=$medialibraryPath; NugetPackageToRestore=$nugetPackageToRestore}
        }
        $Body = $Obj | ConvertTo-JSON
        Write-Host $Body
        	
        $params = @{
        	ContentType = 'application/json'
        	Headers     = @{
        		'authorization' = "token $($GithubToken)"
        		'accept'        = 'application/vnd.github.v3+json'
        	}
        	Method      = 'Post'
        	URI         = $Uri
        	Body        = $Body
        }
        
        Invoke-RestMethod @params -verbose
        Write-Host "Triggering GitHub Action"
        
        Write-Host "Check out the job you triggered at https://github.com/sandbox/therepo/actions/workflows/DeployContentPackage.yml"

        exit
}
Close-Window

Let me break down the script for you 🙂

First, we set some secret stuff. This is for GitHub Packages and triggering GitHubActions. They are all environment variables that are set in the docker-compose file.

$GithubOrganization = $env:GithubOrg

$GithubRepo = $env:GithubRepo

$GithubToken = $env:GithubToken

$ClientSecret = $env:$ClientSecret

We need to find out what website we want to restore/install content to and some other information:

$siteName, $tenantName, $sitePath, $medialibraryPath  = Get-SiteInfo

function Get-SiteInfo {

	Set-Location -Path .
	
	$currentItemId = Get-Item .
    $currentItemPath = $currentItemId.ItemPath 
	$medialibraryItem = Get-Item $currentItemPath"/Media"
	
	$virtualFolderField = [Sitecore.Data.Fields.MultilistField]$medialibraryItem.Fields["AdditionalChildren"]
	$firstInList = $virtualFolderField.GetItems() | Select-Object -first 1
	$mediaItem = Get-Item $firstInList.Id
 
 
	return @($currentItemId.Name, $currentItemId.Parent.Name, $currentItemPath, $mediaItem.ItemPath );

}

Now to some cool stuff, I wanted to get available GitHub Packages (NuGet) for a specific site and return the result as dropdown options.

*We are storing Sitecore Cli itempackages in NuGet packages(at GitHub Packages)

To get info from GitHub Packages, there is this excellent rest api that allows you to query GitHub Packages.

In our scenario, we fetch the packages info, parse the result to JSON, and filter out available NuGet packages for a specific site name. 

$nugetPackages=Get-Packages-BySiteName $siteName

function Get-Packages-BySiteName {

    Param($SiteName)

    $Headers=@{"Accept"="application/vnd.github+json"
    "Authorization"="Bearer $GithubToken"
    "X-GitHub-Api-Version"="2022-11-28"
    }

    $Uri = ('https://api.github.com/orgs/{0}/packages?package_type=nuget' -f $GithubOrganization)

    $options = [Ordered]@{}
    
    Try
    {
       $result = Invoke-WebRequest -UseBasicParsing -H $Headers $Uri |  ConvertFrom-Json
       $result.Where{$_.Name -Match $SiteName}.Name | ForEach-Object -Process {$options +=@{$_=$_}}
    }
    Catch
    {
       
    }

    return $options

}

We also give some proper instructions on making cm and id publicly available by installing localtunnel.

$localHostInstructions=@"
1. Install localtunnel:
-------------------
npm install -g localtunnel
-------------------

2. Open a powershell window run the following for your id host:
-------------------
lt --local-host id.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://blabla-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button

Copy and paste the url to "Your exposed id host"

3. Open a second powershell window to run the following command for your cm host:
-------------------
lt --local-host cm.mysandbox.localhost --local-https --allow-invalid-cert --port 443
-------------------
You should get something like this:
your url is: https://xyasas-81-232-50-200.loca.lt
You also need to browse the url, one time only thing, and hit the "Click to Continue" button

Copy and paste the url to "Your exposed cm host"

"@

When clicking the “Ok button”(VALIDATE & RESTORE), we will collect info from the form and trigger a GitHubAction with event_type “deploy-content-package”. This means we will have a GitHub Action that listens to the event “deploy-content-package”. And keep in mind that we have made CM and ID available publicly. This means that the GitHubAction will have access to our local environment 😉

if($result -ne "cancel"){

        $Uri = ('https://api.github.com/repos/{0}/{1}/dispatches' -f $GithubOrganization, $GithubRepo)

        
        $Obj = [PSCustomObject]@{
        	event_type = "deploy-content-package"
        	client_payload = [PSCustomObject]@{ IdHost=$IdHost; CmHost=$CmHost; ClientSecret=$ClientSecret; SiteTenant=$tenantName; SiteName=$siteName; SitePath=$sitePath; SiteMedialibraryPath=$medialibraryPath; NugetPackageToRestore=$nugetPackageToRestore}
        }
        $Body = $Obj | ConvertTo-JSON
        Write-Host $Body
        	
        $params = @{
        	ContentType = 'application/json'
        	Headers     = @{
        		'authorization' = "token $($GithubToken)"
        		'accept'        = 'application/vnd.github.v3+json'
        	}
        	Method      = 'Post'
        	URI         = $Uri
        	Body        = $Body
        }
        
        Invoke-RestMethod @params -verbose
        Write-Host "Triggering GitHub Action"
        
        Write-Host "Check out the job you triggered at https://github.com/sandbox/therepo/actions/workflows/DeployContentPackage.yml"

        exit
}

And here is what the dialog form looks like:

The final part is the GitHub Action, which will listen to the event “deploy-content-package”. Here we have the GitHub Action:

name: Dispatch event - Deploy backup content package to local cm - Linux
run-name: Dispatch event - Deploy backup package ${{ github.event.client_payload.NugetPackageToRestore }} to local host 

on: 
   repository_dispatch:
    types: [deploy-content-package]


jobs:
  install-nuget:
      name: Install NuGet package  
      runs-on: self-hosted

      steps:
      - name: Checkout SandboxSolution repo
        uses: actions/checkout@v3
        with:
          repository: sandbox/SandboxSolution
          token: ${{ secrets.REPO_PAT }}
          ref: test

      
      - name: Install dot net
        shell: bash
        run: | 
          wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
          sudo dpkg -i packages-microsoft-prod.deb
          rm packages-microsoft-prod.deb

          sudo apt-get update; \
          sudo apt-get install -y apt-transport-https && \
          sudo apt-get update && \
          sudo apt-get install -y dotnet-sdk-3.1
      

      - name : Install Mono
        shell: bash
        run: |
          sudo apt install -y gnupg ca-certificates
          sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
          echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
          sudo apt update
          sudo apt install -y mono-complete
          mono --version

      - name: Install PowerShell 7.2
        shell: bash  
        run: |     
          sudo apt-get update
          sudo apt-get install -y wget apt-transport-https software-properties-common
          wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
          sudo dpkg -i packages-microsoft-prod.deb
          sudo apt-get update
          sudo apt-get install -y powershell

      - name: Install sitecore cli
        shell: bash  
        run: | 
          dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
          dotnet tool restore
 
      - name : Create module.json file
        id: create-json-module
        uses: jsdaniell/create-json@1.1.2
        with:
          name: 'Content.module.json'
          dir: 'SandboxSolution/src/Project/${{ github.event.client_payload.SiteName }}/'
          json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"}]}}'
          

      - name : Install NuGet package   
        shell: pwsh
        run: |
                mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/sandbox/index.json" -UserName ${{secrets.USERNAME}} -Password ${{secrets.NUGET_PACKAGE_TOKEN}}
                mono BackupContent/nuget.exe install ${{ github.event.client_payload.NugetPackageToRestore }} -Source "backupcontent" -OutputDirectory BackupContent -x

            
      - name : Deploy SCS package to CM   
        shell: pwsh
        run: |
                # Login to ID Server
                dotnet sitecore login --client-credentials true --auth ${{github.event.client_payload.IdHost}} --cm ${{github.event.client_payload.CmHost}} --allow-write true --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}"

                # Install package
                dotnet sitecore ser package install --package BackupContent/${{ github.event.client_payload.NugetPackageToRestore }}/Sitecore/${{ github.event.client_payload.NugetPackageToRestore }}.itempackage --cm ${{github.event.client_payload.CmHost}} --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}" -i Project.Content

Let me break it down for you 🙂

We set the event that will trigger the GitHub Action. Notice that we are using a self-hosted GitHub Action. This is not necessary. You can use a “public” GitHub Action.

However, we have the GitHub Actions hosted in our Kubernetes cluster, and that is how you should do it(If you have a running Kubernetes cluster). Check out my previous post on how to set up Actions Runner Controller (ARC) in your Kubernetes cluster – Run Sitecore CLI within your Kubernetes cluster using self-hosted GitHub actions

name: Dispatch event - Deploy backup content package to local cm - Linux
run-name: Dispatch event - Deploy backup package ${{ github.event.client_payload.NugetPackageToRestore }} to local host 

on: 
   repository_dispatch:
    types: [deploy-content-package]


jobs:
  install-nuget:
      name: Install NuGet package  
      runs-on: self-hosted

Since we are running on a “Linux” GitHub Action, we must install Dot Net, Mono, and PowerShell. Dot Net and Powershell need no explanation. Mono is required if you want to execute an “executable” like Nuget.exe and 🙂

And, of course, install Sitecore Cli, the best command tool ever!

      - name: Install dot net
        shell: bash
        run: | 
          wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
          sudo dpkg -i packages-microsoft-prod.deb
          rm packages-microsoft-prod.deb

          sudo apt-get update; \
          sudo apt-get install -y apt-transport-https && \
          sudo apt-get update && \
          sudo apt-get install -y dotnet-sdk-3.1
      

      - name : Install Mono
        shell: bash
        run: |
          sudo apt install -y gnupg ca-certificates
          sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
          echo "deb https://download.mono-project.com/repo/ubuntu preview-focal main" | sudo tee /etc/apt/sources.list.d/mono-official-preview.list
          sudo apt update
          sudo apt install -y mono-complete
          mono --version

      - name: Install PowerShell 7.2
        shell: bash  
        run: |     
          sudo apt-get update
          sudo apt-get install -y wget apt-transport-https software-properties-common
          wget -q "https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb"
          sudo dpkg -i packages-microsoft-prod.deb
          sudo apt-get update
          sudo apt-get install -y powershell

     - name: Install sitecore cli
        shell: bash  
        run: | 
          dotnet nuget add source https://sitecore.myget.org/F/sc-packages/api/v3/index.json --name "Sitecore-Public-Nuget-Feed"
          dotnet tool restore

Because we have itempackages in our NuGet Packages, we need to set up/create a module.json file. Telling what content should be pushed to our local Sitecore environment. Notice how we use github.event.client_payload, which is set( and sent) from the Sitecore SPE script 🙂

      - name : Create module.json file
        id: create-json-module
        uses: jsdaniell/create-json@1.1.2
        with:
          name: 'Content.module.json'
          dir: 'SandboxSolution/src/Project/${{ github.event.client_payload.SiteName }}/'
          json: '{"namespace": "Project.Content", "items":{"includes":[{"name":"Images","path":"${{ github.event.client_payload.SiteMedialibraryPath }}","database": "master"},{"name": "Content","path": "${{ github.event.client_payload.SitePath }}/home","database": "master"},{"name": "Data","path": "${{ github.event.client_payload.SitePath }}/data","database": "master"}]}}'

It’s time to install the “requested” NuGet Package – github.event.client_payload.NugetPackageToRestore. Notice how we use mono to execute nuget.exe 

      - name : Install NuGet package   
        shell: pwsh
        run: |
                mono BackupContent/nuget.exe source Add -Name "backupcontent" -Source "https://nuget.pkg.github.com/sandbox/index.json" -UserName ${{secrets.USERNAME}} -Password ${{secrets.NUGET_PACKAGE_TOKEN}}
                mono BackupContent/nuget.exe install ${{ github.event.client_payload.NugetPackageToRestore }} -Source "backupcontent" -OutputDirectory BackupContent -x

In the final part, the workflow will log in and push the Sitecore content(itempackage) to my local Sitecore environment. And since we have made CM and ID publicly available, it will work like a charm 🙂

      - name : Deploy SCS package to CM   
        shell: pwsh
        run: |
                # Login to ID Server
                dotnet sitecore login --client-credentials true --auth ${{github.event.client_payload.IdHost}} --cm ${{github.event.client_payload.CmHost}} --allow-write true --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}"

                # Install package
                dotnet sitecore ser package install --package BackupContent/${{ github.event.client_payload.NugetPackageToRestore }}/Sitecore/${{ github.event.client_payload.NugetPackageToRestore }}.itempackage --cm ${{github.event.client_payload.CmHost}} --client-id "SANDBOX_Automation" --client-secret "${{ github.event.client_payload.ClientSecret }}" -i Project.Content

SPE, Sitecore Cli, GitHub Actions, and GitHub Packages truly rock!

That’s all for now folks 😊

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.