Map an environment variable in Github Actions - github-actions

I created a GitHub Actions Job with a strategy matrix that creates a set of environment variables.
One of them is machine_architecture which is either 32 or 64.
In most steps I can use it directly i.e. via ${{ machine_architecture }}.
But some steps requires strings like 'i386' vs 'x86_64'. Is there an easy way in github actions to create a map-object that I can use in expressions like:
map_object = { 32: "i386", 64: 'x86_64' }
...
${{ map_object[machine_architecture] }}
If not, what is the idiomatic way in github actions to solve that problem?
PS: I am aware, that I can set environment variables in steps, but the problem is, that these variables are only available for the following steps (i.e. not for usage in "run-on:" tag)

In the meantime I found a solution:
Although GitHub Actions has no syntax for directly creating a Mappings/Objects it can be done indirectly with fromJson():
${{ fromJson('{ 32: "i386", 64: "x86_64" }')[machine_architecture] }}
this fromJson() will create a mapping from int to string. the following []-operator resolves the int type "machine_architecture" to a string type.

Here is a way to do it with JSON and jq. It creates the step output ${{ steps.vars.outputs.arch }} which you can use in later steps.
jobs:
varMap:
strategy:
matrix:
machine_architecture: [32, 64]
runs-on: ubuntu-latest
steps:
- name: Set arch var
id: vars
run: |
echo ::set-output name=arch::\
$(echo '{ "32": "i386", "64": "x86_64" }' | jq -r 'to_entries[] | select(.key=="${{ matrix.machine_architecture }}") | .value')
- name: Test arch var
run: echo "Testing ${{ steps.vars.outputs.arch }}"

I don't need a matrix for this, but I need a map for lookups.
Couldn't get #PeterEvans answer to work with GitHub, so I adapted it slightly:
jobs:
yourJobName:
name: Cool Job
runs-on: ubuntu-latest
steps:
- name: Get Machine Arc
run: |
MACHINE_ARC_MAP=$(cat <<END
{
32: "i386",
64: "x86_64",
999: ${{ secrets.SECRET_ARC }}
}
END
)
TARGET_ARC=$(echo $MACHINE_ARC_MAP | jq -r 'to_entries[] | select(.key=="${{ github.event.inputs.machine_architecture }}") | .value')
echo "TARGET_ARC=$TARGET_ARC" >> $GITHUB_ENV
- name: Echo Selected value
run: |
echo ${env.TARGET_VAR}

How about your "map_object" is actually a file mapping machine_architecture values into the values you need, such:
32=i386
64=x86_64
or any other format you want to keep.
Then, your job can define it in a secondary variable as:
jobs:
FirstJob:
name: job 1
runs-on: .....
steps:
- uses: ....
- name: Define variables
run: |
cat $(cat MAP_OBJECT_FILE_NAME) | grep $(cat machine_architecture)= | > MACHINE_ARCHITECTURE_STRING
From there on, you would have the MACHINE_ARCHITECTURE_STRING variable available for the jobs you need. You could of course do it much more simple concatenating or whatever, but here you maintain the mapping in your code with the mapping file and is escalable.

Related

Having issues with variables and outputs

I started learning Github Actions a few days ago and I am playing around with it. I am having issues with defining outputs and using them in variables or as input values in another workflow step. I am not sure what I am doing wrong.
name: Demo
on:
workflow_dispatch:
jobs:
build:
runs-on: [ ubuntu-latest ]
outputs:
paths: ${{ steps.find_path.outputs.paths }}
steps:
- name: Code Checkout
uses: actions/checkout#v2
- name: Find Paths
id: find_path
run: |
all_paths=$(cat document.txt)
# some code
echo "::set-output name=paths::$(echo ${all_paths})"
- name: List
id: list_path
run: |
var_paths=${{ steps.find_path.outputs.paths }}
for part in ${var_paths[#]}; do
# parent_dir=$(cat...)
# ....
# some code
echo $part
done
I have a text file with some paths in it
/home/ubuntu
/home/ubuntu/docs
/home/ariel/code
I want to use the output from find_path step in another list_path step. There is some other code I didn't include as it's irrelevant to my question. So I defined the paths output in find_path step and it is basically a space-separated string /home/ubuntu /home/ubuntu/docs /home/ariel/code and that's format I want to have.
But I don't know how to use outputs from one step as input value in another step.
Basically,
var_paths=/home/ubuntu /home/ubuntu/docs /home/ariel/code
But when I loop var_paths I get weird errors like no such file or directory.
You're missing the quotes when assigning the output from the step to the variable:
name: Demo
on:
workflow_dispatch:
jobs:
build:
runs-on: [ ubuntu-latest ]
steps:
- name: Code Checkout
uses: actions/checkout#v3
- name: Find Paths
id: find_path
run: |
all_paths=$(cat document.txt)
# some code
echo "::set-output name=paths::$(echo ${all_paths})"
- name: List
id: list_path
run: |
# add quotes here
var_paths="${{ steps.find_path.outputs.paths }}"
for part in ${var_paths[#]}; do
# parent_dir=$(cat...)
# ....
# some code
echo $part
done
Misc comments:
You don't need to declare the outputs on the job level unless the output from a step should also be the output from the job. In your case, you're only using the output between steps.
The checkout action is available in v3

Check whether environment variable is empty

What is the nicest approach to check if environment variable is empty on Github action as a condition to a step? I've tried the trivial approach but it doesn't seem to work.
For example:
name: SimpleWorkflow
on:
push:
env:
MULTI_LINE_ARG: |
ARG1: value
jobs:
update:
runs-on: ubuntu-latest
steps:
- name: Fetching Local Repository
uses: actions/checkout#master
- name: run step if multiline not null
if: ${{env.MULTI_LINE_ARG}} != ""
run: echo multiline not null
No matter how I've tried this I fail to properly check if env is empty.
Updated to new GitHub syntax (as of 2022-12)
Background: The issue of secrets not available to forks is known to GitHub folks, but no concrete activity announced: https://github.community/t/github-workflow-not-running-from-pull-request-from-forked-repository/16379/41?u=koppor
One can use output variables of steps to determine whether a secret is available.
The name of the secret in GitHub is SECRET.
- name: Check secrets presence
id: checksecrets
shell: bash
run: |
if [ "$SECRET" == "" ]; then
echo "secretspresent=NO" >> $GITHUB_OUTPUT
else
echo "secretspresent=YES" >> $GITHUB_OUTPUT
fi
env:
SECRET: ${{ secrets.SECRET}}
- name: run step if secret is present
if: (steps.checksecrets.outputs.secretspresent == 'YES')
run: echo secret is present
Note: Solution adapted from https://github.community/t/if-expression-with-context-variable/16558/6?u=koppor.

How to replace string in expression with GitHub actions

This is my action that returns $TOXENV that looks like this py3.6-django2.2 I'd like to $TOXENV to look like this instead py36-django22 is there any substitute/replace function that I could use to replace . char?
name: CI
on:
workflow_dispatch:
branches: [ master, actions ]
jobs:
demo:
runs-on: ubuntu-latest
strategy:
matrix:
python: [3.6, 3.7, 3.8, 3.9]
django: ['2.2', '3.0']
steps:
- uses: actions/checkout#v2
- uses: actions/setup-python#v1
name: Set up Python ${{ matrix.python }} ${{ matrix.django }}
with:
python-version: ${{ matrix.python }}
- name: python version
env:
TOXENV: "py${{ matrix.python }}-django${{ matrix.django }}"
run:
echo $TOXENV
Another way by using BASH native variable substitution:
- name: python version
env:
TOXENV: "py${{ matrix.python }}-django${{ matrix.django }}"
run: |
TOXENV=${{ env.TOXENV }}
TOXENV=${TOXENV//.} # replace all dots
echo TOXENV=${TOXENV} >> $GITHUB_ENV # update GitHub ENV vars
- name: print env
run: echo ${{ env.TOXENV }}
The idea is to read the GitHub actions expression variable into a BASH variable and do the string manipulation then export or set-output to update in GitHub actions runtime.
I don't think there's an easy way to do this in the env directive of your step when defining the value of TOXENV. The env directive accepts expressions, but the functions that can be used in expressions are limited, with nothing that can replace arbitrary characters. The closest I could find is format(), but that unfortunately requires numbered braces in the target string, which won't work for your situation.
Instead, perhaps you could set the value of TOXENV in the run directive using sed, then add it to the environment:
- name: python version
run:
RAW_TOXENV="py${{ matrix.python }}-django${{ matrix.django }}"
TOXENV=$(echo $RAW_TOXENV | sed 's/\.//')
echo "TOXENV=$TOXENV" >> $GITHUB_ENV

Use dynamic input value for Environment in GitHub Actions workflow job

I'm trying to make a GitHub Actions workflow where one job has a dynamic value to its environment setting. https://docs.github.com/en/actions/reference/environments I have two jobs in this workflow. The first job determines which environment the second job will run in, and this is in turn based on which git branch the Actions job is run from.
This is my naive attempt to make it work, but I get an error which says
(Line: 29, Col: 18): Unrecognized named-value: 'needs'. Located at position 1 within expression: needs.get-environment.outputs.environment_name
name: Environments
on:
push:
workflow_dispatch:
jobs:
get-environment:
runs-on: ubuntu-latest
outputs:
environment_name: ${{ steps.get_environment.outputs.environment_name }}
steps:
- id: get_environment
run: |
if [ "$GITHUB_REF" = "refs/heads/test" ]
then
echo "::set-output name=environment_name::test"
elif [ "$GITHUB_REF" = "refs/heads/qa" ]
then
echo "::set-output name=environment_name::qa"
elif [ "$GITHUB_REF" = "refs/heads/master" ]
then
echo "::set-output name=environment_name::production"
fi
use-environment:
runs-on: ubuntu-latest
needs: [get-environment]
environment: ${{ needs.get-environment.outputs.environment_name }}
steps:
- uses: actions/checkout#v2
- name: Run a one-line script
run: echo ${{ secrets.ENV_DEPENDENT_SECRET }}
Is it possible to achieve what I'm trying to do? My end goal is to have a single workflow file for three different app environments (test, QA and prod), where each app environment uses a separate Actions environment. (terminology gets confusing, I know)
I actually had to solve this issue for the place I work. You can use a matrix with the list of your env names and dynamically set the environment name. Take a look at the link for the workflow file.
Have you tried
> use-environment:
> runs-on: ubuntu-latest
> needs: [get-environment]
> environment:
> name: ${{ needs.get-environment.outputs.environment_name }}

GitHub Actions expression functions: string manipulation?

In a GitHub Actions workflow definition file, there's a set of built-in functions that you can use in expressions.
For example: ${{ toJson(github) }}
Are there any string manipulation functions that can be used in an expression, such as toLowerCase?
The documentation page doesn't mention any. However, I'm wondering if Github uses some sort of standard templating / expression eval library under the hood which has provides a larger set of functions out of the box.
Impossible. GitHub expressions doesn't allow string modification, only concatenation.
You could do almost the same with a custom step in a build job, but this means that you won't be able to use that variable everywhere (for example "processed" environment name is out of the question).
env:
UPPERCASE_VAR: "HELLO"
steps:
- id: toLowerCase
run: INPUT=${{ env.UPPERCASE_VAR }} echo "::set-output name=lowerCaseValue::${INPUT,,}"
- run: echo ${{steps.toLowerCase.outputs.lowerCaseValue}}
I wanted to replace some chars in git version strings and was able to make a step like so:
- name: prepare version string
id: prep_version
run: |
export test_version=$(echo ${{ steps.tag_version.outputs.new_version }} | sed 's/[^0-9,a-z,A-Z]/\-/g')
echo ::set-output name=version::$test_version
that worked pretty good for me... so really we have anything that we could put on a cmd line
I usually start by setting all global variables that I will use throughout the workflow.
jobs:
variables:
outputs:
tag_name: ${{ steps.var.outputs.tag_name}}
runs-on: "ubuntu-latest"
steps:
- name: Setting global variables
uses: actions/github-script#v6
id: var
with:
script: |
core.setOutput('tag_name', '${{ github.head_ref }}'.toLowerCase().replaceAll(/[/.]/g, '-').trim('-'));
Usage:
deploy:
needs: [build, variables]
runs-on: [ self-hosted, preview ]
env:
TAG_NAME: ${{ needs.variables.outputs.tag_name }}
step:
-name: Echo variables
run: |
echo ${{ needs.variables.outputs.tag_name }}
echo ${{ env.TAG_NAME}}