Reference the runner context in job's env clause - github-actions

Here's a GitHub Actions workflow file for a Python project named spam:
name: PyInstaller
on:
[...]
jobs:
create_release:
[...]
make_artifact:
needs: create_release
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
runs-on: ${{ matrix.os }}
env:
ARTIFACT_PATH: dist/spam.zip
ARTIFACT_NAME: spam-${{ runner.os }}.zip
steps:
[...]
When this runs, the workflow fails at startup with this:
The workflow is not valid. [...]:
Unrecognized named-value: 'runner'. Located at position 1 within expression: runner.os
I'm attempting to use the os attribute of the runner context. This SO Q&A mentions that the env context can only be used in specific places, so I suspect something similar is happening here. However, I can't find any official documentation addressing this.
Is there any way to reference the runner context to set an environment variable within the env clause of a job, as shown above?
I'm looking for a way to set the environment variable for all steps in the job, so an env inside a step item won't do.
The workaround I've got for now is to add a step specifically to set environment variables:
steps:
- name: Setup environment
run: |
echo "ARTIFACT_NAME=spam-${{ runner.os }}.zip" >> $GITHUB_ENV
however this only works on the Linux runner.

If you scroll down a bit further in the GitHub Actions docs you linked, there's an example workflow printing different contexts to the log.
- name: Dump runner context
env:
RUNNER_CONTEXT: ${{ toJson(runner) }}
I set up a test repo with a workflow demonstration:
on: push
jobs:
one:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- 'ubuntu-latest'
- 'windows-latest'
- 'macos-latest'
steps:
- name: Dump runner context
env:
RUNNER_CONTEXT: ${{ toJson(runner) }}
run: echo "$RUNNER_CONTEXT"
- name: Get runner OS
env:
RUNNER_OS: ${{ runner.os }}
run: echo "$RUNNER_OS"
- name: Create file with runner OS in name
env:
OS_FILENAME: 'spam-${{ runner.os }}.zip'
run: |
echo "OS_FILENAME=spam-${{ runner.os }}.zip" >> $GITHUB_ENV
touch "./${{ env.OS_FILENAME }}"
touch blah.txt
- name: List created file
run: ls -l "./${{ env.OS_FILENAME }}"
It looks like you can also set and access env in steps, and those persist across workflow steps. For example, I set the environment variable $OS_FILENAME in step 3 using the echo syntax, and reference it in step 4. This works across all the OS options offered on GitHub Actions.
Note that the GitHub Actions docs state that "Environment variables must be explicitly referenced using the env context in expression syntax or through use of the $GITHUB_ENV file directly; environment variables are not implicitly available in shell commands.". Basically, it means you can't implicitly refer to env variables like $FOO and instead must refer to them as ${{ env.FOO }} in shell commands.
So for your scenario, does it satisfy your requirements if you set $ARTIFACT_NAME in the first step of the job? I wonder if the reason might be that the runner context isn't created until the first step - I'm not sure I have a way of testing this.

Related

Reference a variable in `uses` when pointing to a path to a container

I have a workflow yml file. At the top I have a section above where the jobs are defined to make them global across jobs:
env:
DBT_REPO: ${{ vars.DBT_REPO }}
This var is a repo variable and I have confirmed it is already set. Pretend it's value is fruits/apples.
Then, in one of my jobs I try to reference this var in a step:
- name: Checkout DBT repo
uses: actions/checkout#v2
with:
repository: ${{ env.DBT_REPO }}
token: ${{ secrets.WORKFLOW_TOKEN }}
ref: ${{ env.DBT_REPO_BRANCH }}
path: ./${{ env.DBT_REPO }}
- name: Run DBT
uses: ./${{ env.DBT_REPO }}/dbt-action
The last line is line 169.
Then, when I try to run this workflow I get an error:
Invalid workflow file: .github/workflows/main.yml#L169
The workflow is not valid. .github/workflows/main.yml (Line: 169, Col: 15): Unrecognized named-value: 'DBT_REPO'. Located at position 1 within expression: DBT_REPO
If I hard code it like so: uses: ./fruits/apples/dbt-action then things work fine. It's only when I attempt to use a variable.
How can I reference a variable in my uses keyword?
This is not possible because the env context is not available to uses. In fact, based on documentation no contexts are available to the uses key.
See: https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability
I believe this is an architectural limitation of GitHub Actions, it appears they want to resolve all workflows/actions at the start of all jobs and thus dynamic resolution isn't possible.

Is it possible to specify image used in steps according to the matrix strategy?

I'm going to setup a workflow on GitHub which uses some docker images to check my package on different platform. I'd like to use the matrix strategy. Then I wrote a workflow like this:
name: Main Workflow
on: [push]
jobs:
test_linux:
runs-on: ubuntu-latest
strategy:
matrix:
image: [ubuntu, fedora]
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Test
uses: docker://${{ matrix.image }}:latest
with:
entrypoint: /bin/bash
args: ./test.sh
This is just a demo workflow. The real one is much more complicated.
However, after I pushed it, GitHub gave me the following error message:
Invalid workflow file: .github/workflows/main.yml#L16
The workflow is not valid. .github/workflows/main.yml (Line: 13, Col: 15): Unrecognized named-value: 'matrix'. Located at position 1 within expression: matrix.image
Actually, I got this error instantly when I pushed codes.
Why? Is the matrix context inaccessible when parsing workflow file? Are there some ways to resolve it?
I tried to export the full docker name to environment in the previous step before "Test", but context env is also inaccessible just like the context matrix. I got nearly the same error. The only different is:
Unrecognized named-value: 'env'...
Now I have to call docker run manually in run scripts like
steps:
- name: Test
run: docker run --rm -v $(pwd):/code --workdir=/code ${{ matrix.image }}:latest /bin/bash -c "/code/test.sh"
It did work but I still prefer to use the uses entry.

Getting empty value for output from a previous job in a new one in a GitHub Action

on: pull_request
jobs:
job1:
runs-on: [re-centos7]
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "test=world1" >> $env:GITHUB_OUTPUT
- id: step2
run: echo "test=world" >> $env:GITHUB_OUTPUT
job2:
runs-on: [re-centos7]
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}}
output
I'm trying to use the job's output in the next job. I'm trying to use github output for this but I see it's getting an empty value for the output. Can someone help? Thanks!
output is empty for ${{needs.job1.outputs.output1}}
I'm using self-hosted runners which are integrated with Kubernetes cluster
The first thing to check is the version of your self-hosted runners: The deprecation of set-output was mentioned recently (oct. 2022)
If you are using self-hosted runners make sure they are updated to version 2.297.0 or greater.
If the version is lower, that would explain the empty values.

How to throw error and stop action on condition in GH action [duplicate]

I'm developing a Github actions workflow. This workflow runs on Linux, Mac, and Windows.
As part of the workflow, I have to check whether 2 environment variables are equal. If they don't - fail the job.
As described here, Github Actions support if: condition:
steps:
- run: # How can I make a cross-platform failure here?
if: ${{ envA }} != ${{ envB }}
How can I make the job fail if the above condition is true?
In the beginning, I thought of a script, but there must be a more elegant way to fail a job.
I'd do run: exit 1. That will simply exit with an exit code of 1, on all three platforms.
Proof that it's cross-platform: https://github.com/rmunn/Testing/runs/220188838 which runs the following workflow:
name: Test exiting on failure
on: [push]
jobs:
build:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout#v1
- name: Try to fail
run: exit 1
- name: Print message if we don't fail
run: echo Should not get here
(An earlier version of this answer recommended "/bin/false", but that would only work on Linux and macOS).
In 2021, there is perhaps a more graceful way to do this:
- name: A/B Check
if: ${{ envA }} != ${{ envB }}
uses: actions/github-script#v3
with:
script: |
core.setFailed('envA and envB are not equivalent!')
Here, we use the github-script action to provide a one liner script that will fail the job. The "A/B Check" step will only run if the condition in the if line is true, so the script will only run in that case, which is what we want.
The nice thing about this approach is that you will get nicely formatted output in the Actions UI in your repo, showing that the "A/B Check" step caused the failure, and why (i.e. "envA and envB are not equivalent").
Note that if you have additional steps in the job after this, and you do NOT want them to run if the A/B check fails, you'll want to use if: success() on them to prevent them from running in that case.
The Github workflow commands docs gives a hint on this.
Toolkit function
Equivalent workflow command
core.setFailed
Used as a shortcut for ::error and exit 1
Given that, you can do the following without using any external workflows.
steps:
- name: A/B Check
if: ${{ envA }} != ${{ envB }}
run: |
echo "::error file={name},line={line},endLine={endLine},title={title}::{message}"
exit 1

Reference output from previous job GH Actions

I am attempting to use GitHub Actions for a complete pipeline, including automatic SemVer versioning (using tags) that I would then like to consume after building my Docker image to tag it with the current version. This is the action that I am using to bump the version, which should have a new_tag output but I cannot reference it, this is what I am trying:
jobs:
setup:
...
version:
needs: [setup]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
with:
fetch-depth: '0'
- name: Bump version and push tag
uses: anothrNick/github-tag-action#1.26.0
id: autoversion
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WITH_V: true
sonar:
...
anchore:
...
docker:
needs: [setup, version]
steps:
...
- name: Build and push
uses: docker/build-push-action#v2
with:
context: .
push: true
tags: ansfire/flaskql:${{ needs.version.autoversion.outputs.new_tag }}
From what I have read using the needs key is supposed to allow one job to access upstream jobs but I cannot get it to access this. Do I need an outputs key in the version stage? Thanks!
Look into this answer, you need to define the outputs in the job creating the outputs, i.e.
jobs:
version:
[...]
outputs:
new_tag: ${{ steps.autoversion.outputs.new_tag }}
docker:
[...] tags: ansfire/flakql:${{ needs.version.outputs.new_tag }}