GitLab CI - passing JSON into a release-cli command with PowerShell - json

I'm trying to pass a JSON structure required to register a new release in GitLab, but I'm not sure how I should escape the double quotes. Here's a release job section from my gitlab-ci.yml:
release:
stage: release
tags:
- windows
rules:
- if: $CI_COMMIT_TAG
variables:
ASSET_LINK_DETAILS: "{\"name\":\"${PACKAGE_NAME}.zip\",\"url\":\"${PACKAGE_REGISTRY_URL}/${PACKAGE_NAME}.zip\"}"
script:
- 'Write-Host "Creating release..."'
- 'Start-Process -FilePath "release-cli" -ArgumentList "create --name `"Release $CI_COMMIT_TAG`" --tag-name `"$CI_COMMIT_TAG`" --asset-link ${env:ASSET_LINK_DETAILS}" -NoNewWindow -Wait'
- 'Write-Host "Finished creating release..."'
Running this job returns an error from the release-cli:
Incorrect Usage: flag provided but not defined: -asset-link
What is the correct way to pass this JSON into the command in PowerShell?

I found the answer in the GitLab release documentation, there's a complete example there. PowerShell can be a PITA...
https://docs.gitlab.com/ee/user/project/releases/release_fields.html#use-a-generic-package-for-attaching-binaries
Here's an example:
- $env:asset = "{`"name`":`"$env:PACKAGE_NAME.zip`",`"url`":`"$env:PACKAGE_REGISTRY_URL/$env:PACKAGE_NAME.zip`"}"
- $env:assetjson = $env:asset | ConvertTo-Json
- Start-Process -FilePath "C:\GitLab\Release-CLI\bin\release-cli" -ArgumentList "create --name `"$CI_COMMIT_TAG`" --description `"Release $CI_COMMIT_TAG`" --tag-name `"$CI_COMMIT_TAG`" --ref `"$CI_COMMIT_TAG`" --assets-link=$env:assetjson" -NoNewWindow -Wait

Related

Creating Dynamic octopusdpeloy relese via github actions

I get the below error when i try to create a release dynamically with octopusdpeloy via github actions
Action code
- name: Create a release in Octopus Deploy
uses: OctopusDeploy/create-release-action#v2
env:
OCTOPUS_API_KEY: ${{ inputs.OCTOPUS_APIKEY }}
OCTOPUS_HOST: "https://octopusdev.fidev.com/"
with:
project: ${{ github.event.repository.name }}
packages: {Deployartifact}:{${{ steps.deploy_setup.outputs.JFROG_VERSION }}}
Run OctopusDeploy/install-octopus-cli-action#v1
Latest version available: 9.1.7
✓ Octopus CLI version found: 9.1.7
⬇️ Downloading Octopus CLI 9.1.7...
📦 Extracting Octopus CLI 9.1.7...
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command "$ErrorActionPreference = 'Stop' ; try { Add-Type -AssemblyName System.IO.Compression.FileSystem } catch { } ; if ((Get-Command -Name Expand-Archive -Module Microsoft.PowerShell.Archive -ErrorAction Ignore)) { Expand-Archive -LiteralPath 'E:\actions-runner\workspace\_temp\2712f7bc-357a-4509-93d3-2318c71651bd.zip' -DestinationPath 'E:\actions-runner\workspace\_temp\86a0af86-765f-4e8f-9e13-9a642c6bd7f2' -Force } else {[System.IO.Compression.ZipFile]::ExtractToDirectory('E:\actions-runner\workspace\_temp\2712f7bc-357a-4509-93d3-2318c71651bd.zip', 'E:\actions-runner\workspace\_temp\86a0af86-765f-4e8f-9e13-9a642c6bd7f2', $true) }"
🐙 Octopus CLI 9.1.7 installed successfully
Run OctopusDeploy/create-release-action#v2
The package argument 'Deployartifact:' does not use expected format of : {Step Name}:{Version}
Exit code: -1
Error: Error: The process 'E:\actions-runner\workspace\_tool\octo\9.1.7\x64\octo.exe' failed with exit code 4294967295
You need to change your packages line to be:
packages: Deployartifact:${{ steps.deploy_setup.outputs.JFROG_VERSION }}
The packages reference does not need the extra braces, just use either PackageID:version or StepName:PackageId:Version.

Github Actions - How to handle a ) in json when passing it across steps

I am querying the github api and then using jq to parse some values from the result
- uses: octokit/request-action#v2.x
id: get_in_progress_workflow_run
with:
route: GET /repos/myorg/myrepo/actions/runs?page=1&per_page=20
env:
GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
- name: Get waiting pull_request_id
id: get_pr_id
run: |
prId="$(echo '${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq '.workflow_runs[] | first(select(.status=="in_progress")) | .pull_requests[0] | .number')";
echo "prId=$prId" >> "$GITHUB_OUTPUT";
This works fine unless the json result from the first step contains a closing parenthesis. When this happens the command substitution get's closed and I get an error about the next line of json being an unrecognized command.
line 1055: timestamp:: command not found
and line 1055
"head_commit": {
"id": "67fb50d15527690eesasdaddc4425fdda5d4e1eba8",
"tree_id": "37df61a25863dce0e3aec7a61df928f53ca64235",
"message": "message with a )",
"timestamp": "2023-01-05T20:27:05Z",
Is there any way to avoid this? I have tried stripping out the ) but I find that no matter how I try to print json from the github context into bash it errors out before I can do anything with it. And there doesn't appear to be a way to do string substitution from the github context.
For instance even just assigning the string to a variable fails with the same error
- name: Get waiting pull_request_id
id: get_pr_id
run: |
json='${{ steps.get_in_progress_workflow_run.outputs.data }}';
fails with
syntax error near unexpected token `)'
${{ .. }} does string interpolation before the shell gets to see anything, so any special character in there can mess up your shell script. It's also a vector for shell injection.
To fix both, set the value in the environment first, and then reference it:
- name: Get waiting pull_request_id
id: get_pr_id
env:
data: ${{ steps.get_in_progress_workflow_run.outputs.data }}
run: |
prId="$(echo "$data" | jq '
.workflow_runs[]
| first(select(.status=="in_progress"))
| .pull_requests[0].number
')"
echo "prId=$prId" >> "$GITHUB_OUTPUT"
Alternatively, you can use the GitHub CLI to make the request without an additional action, and use the --jq parameter instead of stand-alone jq:
- name: Get waiting pull_request_id
id: get_pr_id
env:
GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
run: |
id=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs" \
--method GET \
--raw-field per_page=20 \
--jq '
.workflow_runs[]
| first(select(.status=="in_progress"))
| .pull_requests[0].number
')
echo "prId=$id" >> "$GITHUB_OUTPUT"

Install problem MiKTeX under GitHub Actions

Since around October 16 2022 we have problems with installing MiKTeX under GitHub Actions.
The error we get is:
Run ./miktexsetup_standalone --local-package-repository=C:/miktex-repository \
./miktexsetup_standalone --local-package-repository=C:/miktex-repository \
--package-set=essential \
--shared \
install
shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
initexmf.exe: The executed process did not succeed.
initexmf.exe: Data: fileName="C:\Program Files\MiKTeX\miktex\bin\x64\initexmf.exe", exitCode="1"
Error: Process completed with exit code 1.
The procedure followed in GitHub Actions consists of a few steps:
Step 1:
- name: Download MikTex (Windows)
run: |
$wc = New-Object System.Net.WebClient;
$maxAttempts=5;
$attemptCount=0;
Do {
$attemptCount++;
Try {
$wc.DownloadFile("https://ctan.math.illinois.edu/systems/win32/miktex/setup/windows-x64/miktexsetup-5.1-x64.zip","miktexsetup-5.1-x64.zip")
} Catch [Exception] {
Write-Host $_.Exception | format-list -force
}
} while (((Test-Path "miktexsetup-5.1-x64.zip") -eq $false) -and ($attemptCount -le $maxAttempts))
shell: pwsh
if: matrix.config.os == 'windows-latest'
Step 2:
- name: Extract MikTex zip (Windows)
shell: bash
run: |
unzip miktexsetup-5.1-x64.zip
if: matrix.config.os == 'windows-latest'
Step 3:
- name: Download MikTex packages (Windows)
shell: bash
run: |
./miktexsetup_standalone --verbose \
--local-package-repository=C:/miktex-repository \
--remote-package-repository="https://ctan.math.illinois.edu/systems/win32/miktex/tm/packages/" \
--package-set=essential \
download
if: matrix.config.os == 'windows-latest'
Step 4 (the failing step):
- name: Install MikTex packages (Windows)
shell: bash
run: |
./miktexsetup_standalone --local-package-repository=C:/miktex-repository \
--package-set=essential \
--shared \
install
if: matrix.config.os == 'windows-latest'
Step 5:
- name: Setting MikTex paths (Windows)
shell: bash
run: |
echo "C:/Program Files/MiKTeX/miktex/bin/x64/" >> $GITHUB_PATH
export PATH="/c/Program Files/MiKTeX/miktex/bin/x64/:$PATH"
echo "Configuring MiKTeX to install missing packages on the fly"
initexmf --admin --verbose --set-config-value='[MPM]AutoInstall=1'
if: matrix.config.os == 'windows-latest'
Any ideas how to solve this problem / what can be the problem?
Based on discussions in https://github.com/MiKTeX/miktex/discussions/1204 and https://github.com/MiKTeX/miktex/issues/1213 the solution is to replace the MiKTeX version's zip file
from:
miktexsetup-5.1-x64.zip
to:
miktexsetup-5.2.0+b8f430f-x64.zip

Unable to extract specific value from Azure Key vault Secret using az keyvault secret show command

I am new to AzureCLI script. I request some guidance here:
Please find the secret(masked value) from my Azure key vault.
{
"**attributes**": {
"created": "2021-10-23T04:26:19+00:00",
"enabled": true,
"id": "https://app-kv-axderfctt.vault.azure.net/secrets/connectionstr/cbcncbc",
"name": "connectionstr",
"**value**": "\"{\\\"eventHubNameSpace\\\":\\\"Pji-11111111-132d-46f7-af0f-aq1234\\\",\\\"eventHubName\\\":\\\"pipeline-swswsw-fb95-4e16-8364-oiu8ikji\\\",\\\"**ConnectionString**\\\":\\\"Endpoint=sb://pji-uit8tihj-132d-46f7-af0f-89897899.servicebus.windows.net/;SharedAccessKeyName=343456-ddddd-4720-b3db-oi89087;SharedAccessKey=xxxx+xxxxxxxx=;EntityPath=pipeline-ttrgfrtyh-fb95-4e16-8364-897uy678i\\\"}\""
I am able to get the complete value but not the specific property(ConnectionString) using below command
CONNECTION_STRING=$(az keyvault secret show --vault-name ${{ parameters.KeyVault }} --name ${{ parameters.SecretName}}--query 'value.ConnectionString')
echo "$CONNECTION_STRING"
I get below error:
ERROR: incorrect usage: [Required] --value VALUE | --file PATH
Please advise on how to extract ConnectionString value excluding EntityPath by escaping slashes \\\.
Thanks in advance.
I resolved this issue using jq library
task: Bash#3
displayName: "Parse WarmRepo Connection String"
inputs:
targetType: 'inline'
script: |
# Parse WarmRepo Connection String
echo 'Parse WarmRepo Connection String'
#Refer https://stedolan.github.io/jq/ to know about jq queries
PipelineConnectionString=`jq -r '.ConnectionString' <<<'$(${{ parameters.StreamingPipelineName }}-EventHub-connectionString)'`

ERROR: Job failed: build directory needs to be an absolute path

Installed Gitlab runner on windows
Registered
configured a test repo with test gitlab ci yaml configuration file
CI/CD pipelines shows error
ERROR: Job failed: build directory needs to be an absolute path
Here is the .gitlab-ci.yml :
image: busybox:latest
before_script:
- echo "Before script section"
- echo "For example you might run an update here or install a build dependency"
- echo "Or perhaps you might print out some debugging details"
after_script:
- echo "After script section"
- echo "For example you might do some cleanup here"
build1:
stage: build
script:
- echo "Do your build here"
test1:
stage: test
script:
- echo "Do a test here"
- echo "For example run a test suite"
test2:
stage: test
script:
- echo "Do another parallel test here"
- echo "For example run a lint test"
deploy1:
stage: deploy
script:
- echo "Do your deploy here"
I don't know what type of runner you have, in my case I had the same issue with docker-ssh runner. A workaround for me was to specify build_dir in config.toml
[[runners]]
name = "jira-ssh-runner"
executor = "docker-ssh"
builds_dir = "/home/gitlab-runner/build"
On windows it works with double slashes in the path
[[runners]]
name = "development-server-runner"
url = "https://*****.com"
token = "n****tq"
builds_dir = "E:\\00-gitlab-build"
executor = "shell"
shell = "powershell"
[runners.custom_build_dir]
enabled = true
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]