Why is command output not passed to next command, github action? - github-actions

I'm trying to set the output from a command to an other command in a github action in the following way:
- name: 'Checkout source code'
uses: actions/checkout#v2
with:
fetch-depth: 0
- name: 'Create delta packages for new, modified or deleted metadata'
run: |
mkdir changed-sources
echo "DIFF=$(git rev-parse --short origin/dev)" >> $GITHUB_ENV
echo "${{ env.DIFF }}"
sfdx sgd:source:delta --to "HEAD" --from ${{ env.DIFF }} --output changed-sources/ --generate-delta --source force-app/
But for some reason it's not working, I receive the following output:
steps:
mkdir changed-sources
echo "DIFF=$(git rev-parse --short origin/dev)" >> $GITHUB_ENV
echo ""
sfdx sgd:source:delta --to "HEAD" --from --output changed-sources/ --generate-delta --source force-app/
What I'm I missing, any help would be highly appreciated!

According to the official GitHub documentation:
If you generate a value in one step of a job, you can use the value in subsequent steps of the same job by assigning the value to an existing or new environment variable and then writing this to the GITHUB_ENV environment file.
In your example, you are using the env variable in the same step.
You should therefore access the env variable in another step instead of in the same one.
Example:
- name: 'Checkout source code'
uses: actions/checkout#v2
with:
fetch-depth: 0
- name: 'Create delta packages for new, modified or deleted metadata'
run: |
mkdir changed-sources
echo "DIFF=$(git rev-parse --short origin/dev)" >> $GITHUB_ENV
- name: NEW STEP
run: |
echo "${{ env.DIFF }}"
sfdx sgd:source:delta --to "HEAD" --from ${{ env.DIFF }} --output changed-sources/ --generate-delta --source force-app/

Related

How to generate a new string by combine github action workflow variables

I need to generate environment variable from ${{ github.event.head_commit.message }}, ${{ github.sha }} and here is what I tried so far
- name: Setup env var
run: |
echo MESSAGE_SHA=${{ github.event.head_commit.message }}-${{ github.sha }}) >> $GITHUB_ENV #only the dash and the SHA is print out
echo ${{ env.MESSAGE_SHA }} #Nothing prints out
The concatenation for the commit message and commit SHA with the dash is not working at all. How can I solve this issue?
I made it work here using this workflow implementation.
For what I observed, there are 2 issues in your implementation:
First issue
According to the documentation, when setting an environment variable, you can't access the variable in the same step.
You can make an environment variable available to any subsequent steps in a workflow job by defining or updating the environment variable and writing this to the GITHUB_ENV environment file. The step that creates or updates the environment variable does not have access to the new value, but all subsequent steps in a job will have access. The names of environment variables are case-sensitive, and you can include punctuation. For more information, see "Environment variables."
Therefore, your implementation should instead looks like this:
- name: Setup env var
run: |
echo MESSAGE_SHA='${{ github.event.head_commit.message }}'-'${{ github.sha }}' >> $GITHUB_ENV
- name: Check env var
run: |
echo ${{ env.MESSAGE_SHA }}
However, that wouldn't be enough depending the context, due to the second point.
Second issue
The ${{ github.event.head_commit.message }} variable can be a multiline variable, in that case, you would need to create the variable with another syntax before adding it to the $GITHUB_ENV.
The solution I came with was using this implementation:
- name: Setup env var
run: |
MESSAGE=$(cat << EOF
'${{ github.event.head_commit.message }}'
-'${{ github.sha }}'
EOF
)
echo MESSAGE_SHA=$MESSAGE >> $GITHUB_ENV
Therefore, combining both concepts above, your workflow implementation should look like this to achieve what you want:
- name: Setup env var
run: |
MESSAGE=$(cat << EOF
'${{ github.event.head_commit.message }}'
-'${{ github.sha }}'
EOF
)
echo MESSAGE_SHA=$MESSAGE >> $GITHUB_ENV
- name: Check env var
run: |
echo ${{ env.MESSAGE_SHA }}
Note that using echo MESSAGE_SHA='${{ github.event.head_commit.message }}'-'${{ github.sha }}' >> $GITHUB_ENV could work if you can always guaranty the commit message is a single line.
EDIT:
This syntax also works to set multilines outputs (and looks easier to use):
run: |
echo "TEST=first line \
second line \
third line" >> $GITHUB_OUTPUT

`GITHUB_PULL_REQUEST_BASE_REF: parameter null or not set` error in github actions

I am getting this error while trying to set a github actions. My goal is to set up a github actions that uses another template for linting and fixing SQL. Here is my github folder.
The models folder contains a single sql file (with .sql file extention). The content of the sql folder is an sql file testing.sql with the query: select a,b,c, document as doc from table.
The workflow file contains the following yml file:
on:
pull_request:
jobs:
test-check:
name: runner / sqlfluff (github-check)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: yu-iskw/action-sqlfluff#v3
id: lint-sql
with:
github_token: ${{ secrets.github_token }}
reporter: github-pr-review
sqlfluff_version: "1.2.0"
sqlfluff_command: "fix" # Or "lint"
config: "${{ github.workspace }}/.sqlfluff"
paths: '${{ github.workspace }}/models'
- name: 'Show outputs (Optional)'
shell: bash
run: |
echo '${{ steps.lint-sql.outputs.sqlfluff-results }}' | jq -r '.'
echo '${{ steps.lint-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'
The .sqlfluff file contains a default configuration from the following site: sqlfulff.
The workflow run is throwing the following error which I couldn't quite figure out:
I don't know what the line 15: GITHUB_PULL_REQUEST_BASE_REF: parameter null or not set means in the error. I would be glad if anyone can help with the error.
It is a parameter used by yu-iskw/action-sqlfluff action.yml in its entrypoint.sh.
SQL_FILE_PATTERN="${FILE_PATTERN:?}"
SOURCE_REFERENCE="origin/${GITHUB_PULL_REQUEST_BASE_REF:?}"
changed_files=$(git diff --name-only --no-color "$SOURCE_REFERENCE" "HEAD" -- "${SQLFLUFF_PATHS:?}" |
grep -e "${SQL_FILE_PATTERN:?}" |
xargs -I% bash -c 'if [[ -f "%" ]] ; then echo "%"; fi' || :)
Set it to the remote branch parameter github_base_ref you want to compare to (main for instance).
In your case:
on:
pull_request:
jobs:
test-check:
name: runner / sqlfluff (github-check)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: yu-iskw/action-sqlfluff#v3
id: lint-sql
with:
github_token: ${{ secrets.github_token }}
reporter: github-pr-review
sqlfluff_version: "1.2.0"
sqlfluff_command: "fix" # Or "lint"
config: "${{ github.workspace }}/.sqlfluff"
paths: '${{ github.workspace }}/models'
github_base_ref: "main" <========================
- name: 'Show outputs (Optional)'
shell: bash
run: |
echo '${{ steps.lint-sql.outputs.sqlfluff-results }}' | jq -r '.'
echo '${{ steps.lint-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'
(Do not include the <======... part, only the github_base_ref: "main")

setting output variables on Windows using cmd.exe as the shell

How can i set output variables when using shell: cmd on Windows? My repo is hosted at Github (not Gitlab).
The following is based on this accepted answer
jobs:
job1:
runs-on: my-windows-machine
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: |
echo '::echo::on'
echo "::set-output name=test::hello"
shell: cmd
- id: step2
run: |
echo '::echo::on'
echo "::set-output name=test::world"
shell: cmd
job2:
runs-on: my-windows-machine
needs: job1
steps:
- run: echo ok: ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}
shell: cmd
The echo stmt in job2 only shows ok: string if shell: cmd is used in the steps in job1.
As the OP Andrew concludes in the comments, output vars just is not supported with cmd.exe.
I went ahead and broke up the steps in my workflow to use shell: cmd for the things that need to be processed by cmd.exe and created a separate step (using the default shell) just to set the output vars.
As an alternative, you can see in "Github Actions, how to share a calculated value between job steps?" the $GITHUB_OUTPUT command, which can be used to define outputs for steps. The outputs can then be used in later steps and evaluated in with and env input sections.
You can see it used in "How can I use a GitHub action's output in a workflow?".
Note: the older ::set-output command has now (Oct. 2022) been deprecated.
name: 'Hello World'
runs:
using: "composite"
steps:
- id: random-number-generator
run: echo "name=random-id::$(echo $RANDOM)" >> $GITHUB_OUTPUT
shell: bash
...
jobs:
test-job:
runs-on: self-hosted
steps:
- name: Call Hello World
id: hello-world
uses: actions/hello-world-action#v1
- name: Comment
if: ${{ github.event_name == 'pull_request' }}
uses: actions/github-script#v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'Output - ${{ steps.hello-world.outputs.random-number }}'
})

Github Actions scheduled nightly job only if there have a been a change (Windows)

I have a nightly job running and I would like it to run only if there has been a commit during the past 24 hours
I have seen this question but it is a linux command. I am looking to achieve the same thing on a Windows self-hosted environment so that command does not work.
I tried the solution there but there is not equivalent to the test command and --after doesn't exist
check_date:
runs-on: self-hosted
name: Check latest commit
outputs:
should_run: ${{ steps.should_run.outputs.should_run }}
steps:
- uses: actions/checkout#v3.0.0
- name: Print Latest Commit
run: echo ${{ github.sha }}
- id: should_run
continue-on-error: true
name: Check if latest commit is less than a day
if: ${{ github.event_name == 'schedule' }}
run: test -z $(git rev-list --after="24 hours" ${{ github.sha }}) && echo "::set-output name=should_run::false"
but I get this error message
Run test -z $(git rev-list --after="24 hours" db8f6566733fd7240baaa51607783d54305efa7d)
test : The term 'test' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At D:\workspace\_temp\4d4d15eb-e2be-422e-a826-44984f2860dc.ps1:2 char:1
+ test -z $(git rev-list --after="24 hours" db8f6566733fd7240baaa5160 ...
+ ~~~~
+ CategoryInfo : ObjectNotFound: (test:String) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : CommandNotFoundException
Error: Process completed with exit code 1.
One issue that I had was that my git version was too old (2.33).
--after was introduced from 2.35
The other one, I "hacked" it by using a script. Not a very elegant solution but it works..
I did put this into a batch file and call it.
#echo off
setlocal enabledelayedexpansion
set result=
for /f "tokens=*" %%i in ('git rev-list %~1 --after="24 hours"') do (
set result=%%i
)
if "!result!"=="" (
echo ::set-output name=should_run::false
) else (
echo ::set-output name=should_run::true
)
And use it like this:
check_date:
runs-on: self-hosted
name: Check latest commit
outputs:
should_run: ${{ steps.should_run.outputs.should_run }}
steps:
- uses: actions/checkout#v3.0.0
- name: Print Latest Commit
run: echo ${{ github.sha }}
- id: should_run
continue-on-error: true
name: Check if latest commit is less than a day
shell: cmd
run: ci_check_last_commit.bat ${{ github.sha }}

Read JSON file in Github Actions

I want to read a JSON file and use a property in a string in a Github Actions YAML file. How do I do this?
(I want the version of the package.json)
Use the built-in fromJson(value) (see here: https://docs.github.com/en/actions/learn-github-actions/expressions#fromjson)
Reading a file depends on the shell you're using. Here's an example for sh:
name: Test linux job
on:
push
jobs:
testJob:
name: Test
runs-on: ubuntu-latest
steps:
- id: set_var
run: |
content=`cat ./path/to/package.json`
# the following lines are only required for multi line json
content="${content//'%'/'%25'}"
content="${content//$'\n'/'%0A'}"
content="${content//$'\r'/'%0D'}"
# end of optional handling for multi line json
echo "::set-output name=packageJson::$content"
- run: |
echo "${{fromJson(steps.set_var.outputs.packageJson).version}}"
Multi line JSON handling as per https://github.community/t5/GitHub-Actions/set-output-Truncates-Multiline-Strings/td-p/37870
GitHub issue about set-env / set-output multi line handling: https://github.com/actions/toolkit/issues/403
Below is a version of the example from Official GHA Docs that includes two changes:
Loads json from a file (./your.json)
Removes newline characters (Source)
Uses fromJson to parse the output and set a matrix variable.
name: build
on: push
jobs:
job1:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- id: set-matrix
run: |
JSON=$(cat ./your.json)
echo "::set-output name=matrix::${JSON//'%'/'%25'}"
job2:
needs: job1
runs-on: ubuntu-latest
strategy:
matrix: ${{fromJson(needs.job1.outputs.matrix)}}
steps:
- run: build
on: [push, pull_request]
name: Build
jobs:
build:
name: Example
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
with:
path: './'
- run: |
echo "`jq '.base_config[0].value="Alpha-21"' config.json `" > config.json
echo "`jq '.base_config[1].value="1.2.14"' config.json`" > config.json
echo "`jq '.base_config[2].value="29/12/2020"' config.json `" > config.json
- uses: EndBug/add-and-commit#v6
with:
message: 'Add the version and date'
add: '*.json --force'
cwd: './'
token: ${{ secrets.TOKEN }}
Use a multi line environment variable:
- run: |
echo 'PACKAGE_JSON<<EOF' >> $GITHUB_ENV
cat ./package.json >> $GITHUB_ENV
echo 'EOF' >> $GITHUB_ENV
- run: |
echo '${{ fromJson(env.PACKAGE_JSON).version }}'
This avoids any need for escaping.
Inspired by answer from #dastrobu which adds key/val to $GITHUB_ENV and using jq to transform/minify package.json to a single line:
- run: echo "PACKAGE_JSON=$(jq -c . < package.json)" >> $GITHUB_ENV
- run: echo '${{ fromJson(env.PACKAGE_JSON).version }}'
I once used this to get the value from the json data. Hope this helps
- name: fetch the json value
run: |
githubjson=`cat $GITHUB_EVENT_PATH`
echo $githubjson
number=`echo $(jq -r '.number' <<< "$githubjson")`
PRTitle=`echo $(jq -r '.pull_request.title' <<< "$githubjson")`
PRUrl=`echo $(jq -r '.pull_request.html_url' <<< "$githubjson")`
PRBody=`echo $(jq -r '.pull_request.body' <<< "$githubjson")`
With Powershell:
- name: Read json
id: read-json
shell: pwsh
run: |
$json = Get-Content yourfile.json | ConvertFrom-Json
echo "::set-output name=prop::$(echo $json.prop)"
- run: echo ${{ steps.read-json.outputs.prop}}
You can easily use the Script action for this.
- name: "Read JSON"
uses: actions/github-script#v6
id: check-env
with:
result-encoding: string
script: |
try {
const fs = require('fs')
const jsonString = fs.readFileSync('./dir/file.json')
var apps = JSON.parse(jsonString)
} catch(err) {
core.error("Error while reading or parsing the JSON")
core.setFailed(err)
}