Options to have variables in GitHub Actions - github-actions

I know that I can use Runner's Linux environment variables in GitHub Actions.
Do I have any other options to have variables and use them in workflow steps?

This is how variables are designed to work in GitHub Actions. I mean declared variables are mapped to env variables, like here:
name: Show env
on:
push:
branches:
- '*'
env:
somevar: 'lastvar'
jobs:
show:
runs-on: ubuntu-latest
steps:
- name: Is variable exported?
run: |
echo "${{ env.somevar }}"
However, you can't use them everywhere - please check this topic:
env:
pluginId: 'plugin-fn-xml-node'
on:
push:
paths:
- ${{env.pluginId}}/**
- .github/workflows/**
pull_request:
paths:
- ${{env.pluginId}}/**
- '.github/workflows/**'
jobs:
build:
env:
working-directory: ${{env.pluginId}}
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [8.x, 10.x, 12.x]
steps:
This will not work because you can't use workflow variable at job level.
So if you define variable at workflow level you should be able to use it across steps.
I added also dynamically set variable based on documentation
env:
somevar: 'lastvar'
jobs:
show:
runs-on: ubuntu-latest
steps:
- name: Is variable exported?
run: |
echo "${{ env.somevar }}"
echo "action_state=yellow" >> $GITHUB_ENV
- name: PowerShell script
# You may pin to the exact commit or the version.
# uses: Amadevus/pwsh-script#25a636480c7bc678a60bbf4e3e5ac03aca6cf2cd
uses: Amadevus/pwsh-script#v2.0.0
continue-on-error: true
with:
# PowerShell script to execute in Actions-hydrated context
script: |
Write-Host $env:somevar
Write-Host $env:action_state
- name: Read exported variable
run: |
echo "$action_state"
echo "${{ env.action_state }}"

Related

GitHub Actions Environment variables for matrix-based testing

I am struggling with the recent change on how GitHub Action Workflows defines runtime variables, replacing the "set-ouput" approach with environment variables.
Last summer, it took me a couple of hours to figure out the code below working for my requirement. Define a matrix of OS and python versions, so that the CI workflow can create corresponding environments and run the pytests on it.
Is there any chance to get support on how to best transform to the new approach?
env:
# JSON variables (used in our strategy/matrix)
SUPPORTED_PYTHON_VERSIONS: '\"python-version\":[\"3.8\", \"3.9\"]'
SUPPORTED_OPERATING_SYSTEMS: '\"os\":[\"ubuntu-latest\", \"macos-latest\", \"windows-latest\"]'
jobs:
# The set-env job translates the json variables to a usable format for the workflow specifications.
set-env:
runs-on: ubuntu-latest
outputs:
py-os-matrix: ${{ steps.set-matrix-vars.outputs.py-os-matrix }}
# ^ this represents:
# matrix:
# - os: [ubuntu-latest, ...]
# - python-version: [3.7, ...]
os-matrix: ${{ steps.set-matrix-vars.outputs.os-matrix }}
# ^ this represents:
# matrix:
# - os: [ubuntu-latest, ...]
steps:
- id: set-matrix-vars
run: |
echo "::set-output name=py-os-matrix::{${{ env.SUPPORTED_PYTHON_VERSIONS }},${{ env.SUPPORTED_OPERATING_SYSTEMS }}}"
echo "::set-output name=os-matrix::{${{ env.SUPPORTED_OPERATING_SYSTEMS }}}"
test-run:
name: test on ${{ matrix.os }} - ${{ matrix.python-version }}
needs: set-env
strategy:
fail-fast: true
matrix: ${{ fromJson(needs.set-env.outputs.py-os-matrix) }}
Meanwhile, a more elegant way of using the matrix keyword functions is available. My new implementation is much cleaner and, most important, avoids the above-mentioned deprecated function of printing env variables to stdout. However, the point is open how to integrate python versions >=3.10 ...
jobs:
test-run:
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
python-version: [3.8, 3.9]
defaults:
run:
shell: bash
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository code
uses: actions/checkout#v3
- name: Install python
id: setup-python
uses: actions/setup-python#v4
with:
python-version: ${{ matrix.python-version }}
check-latest: true
Every further step which comes after the one with setup-python is executed in the currenlty selected combination of OS and python.

GitHub Actions - How to use environment variable at job level?

I want to use environment variables at the job level. Is there a way to do it?
env:
stageEnv: UAT
jobs:
name: Upload Build
if: ${{ env.stageEnv == 'UAT' }}
steps:
....
I get unrecognized named-value: 'env' error. Tried $stageEnv and ${{ env.stageEnv }}
Note: It works when I access within 'steps', but would like this to be accessible at 'jobs' level.
I'm afraid not, but you can do like this:
env:
stageEnv: UAT
jobs:
build:
name: Build
runs-on: ubuntu-latest
outputs:
stageEnv: ${{ steps.init.outputs.stageEnv }}
steps:
- name: Make environment variables global
id: init
run: |
echo "stageEnv=${{ env.stageEnv }}" >> $GITHUB_OUTPUT
And use it in another job:
upload:
name: Upload build
needs: build
if: ${{ needs.build.outputs.stageEnv == 'UAT' }}
Note this is just an example, and I personally prefer environment variables uppercase and output variables lowercase

Setting env varables for entire workflow [duplicate]

I'd like to define and set environment variable between jobs inside my Github Actions Workflow.
The workflow below is what I've tried but unfortunately the environment variable GIT_PR_SHA_SHORT and E2E_GIT_PR_SHA not working.
Is it possible?
name: Git Pull Request Workflow
on:
workflow_dispatch:
pull_request:
branches:
- master
env:
GIT_PR_SHA: ${{github.event.pull_request.head.sha}}
GIT_PR_SHA_SHORT: "${{ env.GIT_PR_SHA:0:10 }}"
ENV_NAME: test
E2E_GIT_PR_SHA: "${{ env.ENV_NAME }}-${{ env.GIT_PR_SHA_SHORT }}"
jobs:
first-job:
name: Build Docker Image
runs-on: ubuntu-latest
steps:
- name: First Echo Step
run: |
echo "GIT_PR_SHA_SHORT = ${GIT_PR_SHA_SHORT}"
echo "E2E_GIT_PR_SHA = ${E2E_GIT_PR_SHA}"
second-job:
name: Build Docker Image
runs-on: ubuntu-latest
steps:
- name: Second Echo Step
run: |
echo "GIT_PR_SHA_SHORT = ${GIT_PR_SHA_SHORT}"
echo "E2E_GIT_PR_SHA = ${E2E_GIT_PR_SHA}"
You reference a workflow's environment variables with ${{ env.VARIABLE_NAME }} not ${VARIABLE_NAME}. The latter is bash syntax, but these are not shell environment variables, they're workflow environment variables. They're part of the workflow execution, not part of the shell's context.
To reference a workflow environment variable:
name: Git Pull Request Workflow
on:
workflow_dispatch:
pull_request:
branches:
- master
env:
one: 1
two: zwei
three: tres
jobs:
first-job:
runs-on: ubuntu-latest
steps:
- run: |
echo "${{ env.one }}"
echo "${{ env.two }}"
echo "${{ env.three }}"
(I like to use lower-case for my workflow environment variables, and UPPER_CASE for my shell environment variables, so that it's more obvious to me which is which.)
Similarly, this won't work:
env:
GIT_PR_SHA_SHORT: "${{ env.GIT_PR_SHA:0:10 }}"
This is mixing bash syntax :0:10 with the workflow syntax, but the workflow variables are not run through any shell. No virtual machine has been started when the workflow file is parsed, so there's no shell to run things though.
If you wanted to use bash expressions to manipulate the environment, you would need to create a step that runs bash to do that, and you would need to use the ::set-env or ::set-output syntax.
Then you can refer to a step's output using the ${{ steps... }} context.
Unfortunately, passing things between different jobs is trickier, since they run on different virtual machines. You'll need to set variables on the overall workflow itself. You'll need to first ::set-output so that it's visible to the job, then you can raise the visibility from the job to the workflow.
name: Demonstration
on:
push:
branches: [master]
jobs:
first-job:
runs-on: ubuntu-latest
steps:
- id: identify
run: |
# use bash variable expression to get the substring
export GIT_PR_SHA="${{ github.sha }}"
export GIT_PR_SHA_SHORT="${GIT_PR_SHA:0:10}"
echo "::set-output name=git_pr_sha::${GIT_PR_SHA}"
echo "::set-output name=git_pr_sha_short::${GIT_PR_SHA_SHORT}"
outputs:
git_pr_sha: ${{ steps.identify.outputs.git_pr_sha }}
git_pr_sha_short: ${{ steps.identify.outputs.git_pr_sha_short }}
second-job:
needs: first-job
runs-on: ubuntu-latest
steps:
- run: |
echo "${{ needs.first-job.outputs.git_pr_sha }}"
echo "${{ needs.first-job.outputs.git_pr_sha_short }}"
I'd like to add an extension to this since I've had similar difficulties finding how to compute & set environment variables for multi-step use.
Below is a basic example of how to push back to the github environment from within a step if processing is needed to compute an environment variable for later use. You can also update existing variables this same way, not just create new.
name: minimal variable example
on:
push:
env:
MAJOR: "1"
MINOR: "0"
PATCH: "1"
jobs:
vars-example:
runs-on: ubuntu-latest
steps:
- name: only available local variable
run: LOCAL_VERSION=${MAJOR}.${MINOR}.${PATCH}
- name: available across multiple steps
run: echo "GLOBAL_VERSION=${MAJOR}.${MINOR}.${PATCH}" >> $GITHUB_ENV
- name: Vars
run: |
echo LOCAL_VERSION = $LOCAL_VERSION
echo GLOBAL_VERSION = $GLOBAL_VERSION
which results in Vars output of
echo LOCAL_VERSION = $LOCAL_VERSION
echo GLOBAL_VERSION = $GLOBAL_VERSION
shell: /usr/bin/bash -e {0}
env:
MAJOR: 1
MINOR: 0
PATCH: 1
GLOBAL_VERSION: 1.0.1
LOCAL_VERSION =
GLOBAL_VERSION = 1.0.1
You can't use env in an expression under the env element. I don't see another way than using duplicate values instead of an expression.
The env context syntax allows you to use the value of an environment variable in your workflow file. You can use the env context in the value of any key in a step except for the id and uses keys. For more information on the step syntax, see "Workflow syntax for GitHub Actions".
https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#env-context
The following is the newer way of sharing the variables across jobs after the deprication
Have also covered the scenario when the value itself is a variable and needs to be computed at runtime.
The following would be invoked when a push is made master branch
name: Sharing envs across jobs
on:
push:
branches: ['master']
env:
one: onevalue
two: twovalue
three: threevalue
jobs:
job0:
runs-on: ubuntu-latest
steps:
- run: |
echo "${{ env.one }}"
echo "${{ env.two }}"
echo "${{ env.three }}"
job1:
runs-on: ubuntu-latest
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "test=$(date +"%d-%m-%Y")-asdfads223" >> $GITHUB_OUTPUT
- id: step2
run: echo "test=world" >> $GITHUB_OUTPUT
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}

Can I define a global across jobs

I have a workflow with 2 jobs:
on: [push]
jobs:
ssql:
runs-on: ubuntu-latest
### bunch of steps in between ###
- name: Upload data as artifact
uses: actions/upload-artifact#v2
with:
name: ${{ env.GAME }} + "-" + "data"
path: output_data/training-data.csv
env:
GAME: "FunGame"
Rtrain:
runs-on: ubuntu-latest
name: Train model
needs: ssql
steps:
- name: checkout current repo
uses: actions/checkout#v2
- name: Retreive data from ssql job
uses: actions/download-artifact#v2
with:
name: ${{ env.GAME }} + "-" + "data"
env:
GAME: "FunGame"
I had to set env variable $GAME twice, once in each job. Is there any syntax where I can e.g. add this at the top to make it truly global across jobs, e.g. something like:
on: [push]
env:
GAME: "FunGame"
jobs: ...
You can set an env variable exactly as you described at the workflow level.
Just note that it can conflict with env variables set at the job level, or at the step level, if they have the same name. In that case, the variable used is the most specific one: STEP over JOB over WORKFLOW.
Reference 1 + Reference 2
Here is an example of how to use env variables at different levels (without conflict):
name: Environment Workflow
on:
workflow_dispatch:
env:
WORKFLOW_VARIABLE: WORKFLOW
jobs:
job1:
runs-on: ubuntu-latest
env:
JOB_VARIABLE: JOB
steps:
- name: Run Commands with various variables
if: ${{ env.WORKFLOW_VARIABLE == 'WORKFLOW' }}
env:
STEP_VARIABLE: STEP
run: |
echo "Hello World"
echo "This is the $WORKFLOW_VARIABLE environment variable"
echo "This is the $JOB_VARIABLE environment variable"
echo "This is the $STEP_VARIABLE environment variable"
Full workflow implementation (if you want to reproduce)
Previous workflow run display

GitHub Action - Define Workflow Level Environment Variable Between Jobs

I'd like to define and set environment variable between jobs inside my Github Actions Workflow.
The workflow below is what I've tried but unfortunately the environment variable GIT_PR_SHA_SHORT and E2E_GIT_PR_SHA not working.
Is it possible?
name: Git Pull Request Workflow
on:
workflow_dispatch:
pull_request:
branches:
- master
env:
GIT_PR_SHA: ${{github.event.pull_request.head.sha}}
GIT_PR_SHA_SHORT: "${{ env.GIT_PR_SHA:0:10 }}"
ENV_NAME: test
E2E_GIT_PR_SHA: "${{ env.ENV_NAME }}-${{ env.GIT_PR_SHA_SHORT }}"
jobs:
first-job:
name: Build Docker Image
runs-on: ubuntu-latest
steps:
- name: First Echo Step
run: |
echo "GIT_PR_SHA_SHORT = ${GIT_PR_SHA_SHORT}"
echo "E2E_GIT_PR_SHA = ${E2E_GIT_PR_SHA}"
second-job:
name: Build Docker Image
runs-on: ubuntu-latest
steps:
- name: Second Echo Step
run: |
echo "GIT_PR_SHA_SHORT = ${GIT_PR_SHA_SHORT}"
echo "E2E_GIT_PR_SHA = ${E2E_GIT_PR_SHA}"
You reference a workflow's environment variables with ${{ env.VARIABLE_NAME }} not ${VARIABLE_NAME}. The latter is bash syntax, but these are not shell environment variables, they're workflow environment variables. They're part of the workflow execution, not part of the shell's context.
To reference a workflow environment variable:
name: Git Pull Request Workflow
on:
workflow_dispatch:
pull_request:
branches:
- master
env:
one: 1
two: zwei
three: tres
jobs:
first-job:
runs-on: ubuntu-latest
steps:
- run: |
echo "${{ env.one }}"
echo "${{ env.two }}"
echo "${{ env.three }}"
(I like to use lower-case for my workflow environment variables, and UPPER_CASE for my shell environment variables, so that it's more obvious to me which is which.)
Similarly, this won't work:
env:
GIT_PR_SHA_SHORT: "${{ env.GIT_PR_SHA:0:10 }}"
This is mixing bash syntax :0:10 with the workflow syntax, but the workflow variables are not run through any shell. No virtual machine has been started when the workflow file is parsed, so there's no shell to run things though.
If you wanted to use bash expressions to manipulate the environment, you would need to create a step that runs bash to do that, and you would need to use the ::set-env or ::set-output syntax.
Then you can refer to a step's output using the ${{ steps... }} context.
Unfortunately, passing things between different jobs is trickier, since they run on different virtual machines. You'll need to set variables on the overall workflow itself. You'll need to first ::set-output so that it's visible to the job, then you can raise the visibility from the job to the workflow.
name: Demonstration
on:
push:
branches: [master]
jobs:
first-job:
runs-on: ubuntu-latest
steps:
- id: identify
run: |
# use bash variable expression to get the substring
export GIT_PR_SHA="${{ github.sha }}"
export GIT_PR_SHA_SHORT="${GIT_PR_SHA:0:10}"
echo "::set-output name=git_pr_sha::${GIT_PR_SHA}"
echo "::set-output name=git_pr_sha_short::${GIT_PR_SHA_SHORT}"
outputs:
git_pr_sha: ${{ steps.identify.outputs.git_pr_sha }}
git_pr_sha_short: ${{ steps.identify.outputs.git_pr_sha_short }}
second-job:
needs: first-job
runs-on: ubuntu-latest
steps:
- run: |
echo "${{ needs.first-job.outputs.git_pr_sha }}"
echo "${{ needs.first-job.outputs.git_pr_sha_short }}"
I'd like to add an extension to this since I've had similar difficulties finding how to compute & set environment variables for multi-step use.
Below is a basic example of how to push back to the github environment from within a step if processing is needed to compute an environment variable for later use. You can also update existing variables this same way, not just create new.
name: minimal variable example
on:
push:
env:
MAJOR: "1"
MINOR: "0"
PATCH: "1"
jobs:
vars-example:
runs-on: ubuntu-latest
steps:
- name: only available local variable
run: LOCAL_VERSION=${MAJOR}.${MINOR}.${PATCH}
- name: available across multiple steps
run: echo "GLOBAL_VERSION=${MAJOR}.${MINOR}.${PATCH}" >> $GITHUB_ENV
- name: Vars
run: |
echo LOCAL_VERSION = $LOCAL_VERSION
echo GLOBAL_VERSION = $GLOBAL_VERSION
which results in Vars output of
echo LOCAL_VERSION = $LOCAL_VERSION
echo GLOBAL_VERSION = $GLOBAL_VERSION
shell: /usr/bin/bash -e {0}
env:
MAJOR: 1
MINOR: 0
PATCH: 1
GLOBAL_VERSION: 1.0.1
LOCAL_VERSION =
GLOBAL_VERSION = 1.0.1
You can't use env in an expression under the env element. I don't see another way than using duplicate values instead of an expression.
The env context syntax allows you to use the value of an environment variable in your workflow file. You can use the env context in the value of any key in a step except for the id and uses keys. For more information on the step syntax, see "Workflow syntax for GitHub Actions".
https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#env-context
The following is the newer way of sharing the variables across jobs after the deprication
Have also covered the scenario when the value itself is a variable and needs to be computed at runtime.
The following would be invoked when a push is made master branch
name: Sharing envs across jobs
on:
push:
branches: ['master']
env:
one: onevalue
two: twovalue
three: threevalue
jobs:
job0:
runs-on: ubuntu-latest
steps:
- run: |
echo "${{ env.one }}"
echo "${{ env.two }}"
echo "${{ env.three }}"
job1:
runs-on: ubuntu-latest
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "test=$(date +"%d-%m-%Y")-asdfads223" >> $GITHUB_OUTPUT
- id: step2
run: echo "test=world" >> $GITHUB_OUTPUT
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}