I am trying to execute some option steps if the previous step is failing.
I need to download the old artifacts in order to avoid that my Terraform build action is not detecting changes. Therefore I added a diff action to identify if the docker file to build the zip-layer has changed, if there is no change the old artifacts from a previous execution should be downloaded. In some cases the last execution does not contain the artifacts e.g. failure of the jobs. In that case I would like to get the latest version based on the existing docker image.
Note: The code is part of a matrix execution but for simplicity I reduced the action to the problem area.
job_prepare:
.....
job_layers:
needs: job_prepare
runs-on: ubuntu-latest
strategy:
matrix: ${{fromJson(needs.job_prepare.outputs.layer_matrix)}}
steps:
- name: Checkout
uses: actions/checkout#v2
- name: matrix name
run: |
echo $GITHUB_WORKSPACE
echo ${{ matrix.path }}
- uses: technote-space/get-diff-action#v3
id: git_diff
with:
PREFIX_FILTER: ${{ matrix.prefix }}
SUFFIX_FILTER: Dockerfile
- name: LayerDockerBuild
id: layerDocker
if: steps.git_diff.outputs.diff
run: |
docker build ...
docker push ...
- name: Layer via Artifacts
if: (steps.git_diff.outputs.diff == false)
uses: dawidd6/action-download-artifact#v2
with:
workflow: review.yml
name: ${{ matrix.name }}
- name: Layer via Docker
if: steps.git_diff.outputs.diff && ${{ failure() }}
id: layerD
run: |
....
docker pull "docker.pkg.github.com/$REPO_NAME/$IMAGE_ID:$VERSION"
docker run --rm -v $GITHUB_WORKSPACE:/data docker.pkg.github.com/$REPO_NAME/$IMAGE_ID:$VERSION cp /packages/${{ matrix.name }}.zip /data
- name: Upload layer zip
if: ${{ always() }}
uses: actions/upload-artifact#v2
with:
name: ${{ matrix.name }}
path: ${{ matrix.name }}.zip
The problem is basically the logic in the line if: steps.git_diff.outputs.diff && ${{ failure() }}
thanks for your help on any hints how to make the step option when the diff is false and the step ~Layer via Artifacts~ is not failing.
To execute your step if the previous one was not a failure you can just set failure() without brackets. With the if, no need of brackets.
if: steps.git_diff.outputs.diff && failure()
Hope it will help
Related
Given the following sample workflow
name: My workflow
on:
push:
branches:
- 'main'
jobs:
job_1:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout#v3
with:
fetch-depth: 0
- name: Get next version
id: get_next_version
uses: thenativeweb/get-next-version#2.5.0
- name: Echo for new version
if: ${{ steps.get_next_version.outputs.hasNextVersion == 'true' }}
run: echo there is a new version
- name: Echo for no new version
if: ${{ steps.get_next_version.outputs.hasNextVersion != 'true' }}
run: echo there is no new version
job_2:
needs: job_1
if: needs.job_1.steps.get_next_version.outputs.hasNextVersion == 'true'
runs-on: ubuntu-latest
steps:
- name: First step
run: echo job_2 is running
The action get-next-version analyzes my commit and calculates a new version. As you can see in job_1 I can access the calculated result.
job_2 depends on job_1 and should only run if there would be a new version. I tried to access the result in the if statement of job_2 but it seems that didn't work, I might be using the wrong syntax.
I get the echo
there is a new version
but job_2 was skipped. Is there a way to get access to the data of get_next_version.outputs ( I want the fields hasNextVersion and version )?
Yes - it's possible.
Each job can define its output as an output of one of its steps.
The related documentation can be found here
name: My workflow
on:
push:
branches:
- 'main'
jobs:
job_1:
runs-on: ubuntu-latest
# define output for first job forwarding output of hasNextVersionOutput job
outputs:
hasNextVersion: ${{ steps.hasNextVersionOutput.outputs.hasNextVersion }}
steps:
- name: Checkout repository
uses: actions/checkout#v3
with:
fetch-depth: 0
- name: Get next version
id: get_next_version
uses: thenativeweb/get-next-version#2.5.0
# add a step to generate the output that will be read by the job
- name: Generate output
run: echo "hasNextVersion=${{
steps.get_next_version.outputs.hasNextVersion }}" >> $GITHUB_OUTPUT
- name: Echo for new version
if: ${{ steps.get_next_version.outputs.hasNextVersion == 'true' }}
run: echo there is a new version
- name: Echo for no new version
if: ${{ steps.get_next_version.outputs.hasNextVersion != 'true' }}
run: echo there is no new version
job_2:
needs: job_1
# read output directly from job (you cannot access its steps
if: needs.job_1.outputs.hasNextVersion == 'true'
runs-on: ubuntu-latest
steps:
- name: First step
run: echo job_2 is running
I want to use environment variables at the job level. Is there a way to do it?
env:
stageEnv: UAT
jobs:
name: Upload Build
if: ${{ env.stageEnv == 'UAT' }}
steps:
....
I get unrecognized named-value: 'env' error. Tried $stageEnv and ${{ env.stageEnv }}
Note: It works when I access within 'steps', but would like this to be accessible at 'jobs' level.
I'm afraid not, but you can do like this:
env:
stageEnv: UAT
jobs:
build:
name: Build
runs-on: ubuntu-latest
outputs:
stageEnv: ${{ steps.init.outputs.stageEnv }}
steps:
- name: Make environment variables global
id: init
run: |
echo "stageEnv=${{ env.stageEnv }}" >> $GITHUB_OUTPUT
And use it in another job:
upload:
name: Upload build
needs: build
if: ${{ needs.build.outputs.stageEnv == 'UAT' }}
Note this is just an example, and I personally prefer environment variables uppercase and output variables lowercase
I have the below github actions where i am storing the release version in manager job and using it in the deployment with manager-uat and manager-production and a weird thing is happening that i get the output current_version available in manager-uat but not in manager-production whereas i am referring to the same variable. Please can somebody suggest.
manager:
runs-on: ubuntu-latest
outputs:
CURRENT_VERSION: ${{ steps.status.outputs.CURRENT_VERSION }}
NEED_RELEASE: ${{ steps.status.outputs.NEED_RELEASE }}
steps:
- uses: actions/checkout#v2
- <more steps>
- name: manager - Check the current version
working-directory: .
run: |
echo "CURRENT_VERSION=$(python tools/check_version.py manager)" >> $GITHUB_ENV
- id: status
run: |
echo "::set-output name=CURRENT_VERSION::${{ env.CURRENT_VERSION }}"
echo "::set-output name=NEED_RELEASE::${{ env.NEED_RELEASE }}"
manager_uat:
needs: [manager]
if: needs.manager.outputs.NEED_RELEASE == 'true'
uses: ./.github/workflows/cd_mlops.yml
secrets: inherit
with:
version: ${{needs.manager.outputs.CURRENT_VERSION}}
service: manager
envir: uat
manager_production:
needs: [ manager_uat ]
if: needs.manager.outputs.NEED_RELEASE == 'true'
uses: ./.github/workflows/cd_mlops.yml
secrets: inherit
with:
version: ${{needs.manager.outputs.CURRENT_VERSION}}
service: manager
envir: production
You cannot use needs.X if X is not a direct dependency of the job.
So in your case, you need to add manager as a dependency of manager_production like so:
manager_production:
needs: [ manager, manager_uat ]
if: needs.manager.outputs.NEED_RELEASE == 'true'
uses: ./.github/workflows/cd_mlops.yml
# ..etc
We have two runners, one for running production jobs and another for running non production jobs, but I am unable to do that using a workflow level environment variable.
Below is what I have:
name: Workflow file
on:
workflow-dispatch
env:
RUNNER_NAME: ${{ contains(github.ref, 'main') && 'Prod Runner' || 'non-Prod Runner' }}
jobs:
job-run:
runs-on: [${{ env.RUNNER_NAME }}]
needs: ...
steps:
..........
I get the following error message:
Invalid workflow file
You have an error in your yaml syntax on line ###
How do I do this? I don't want to have separate workflow files for prod and non-prod workflows.
For what you can check on this Github Actions ISSUE, it seems it's not possible to use natively env variable on the runs-on job field (yet?).
However, there is a workaround if you configure a variable as output in a previous job, so you would be able to use it afterwards.
Example: runs-on: ${{ needs.setup.outputs.runner }}
In your case, the workflow would look like this:
on:
workflow_dispatch:
jobs:
setup:
runs-on: ubuntu-latest
outputs:
runner: ${{ steps.step1.outputs.runner }}
steps:
- name: Check branch
id: step1
run: |
if [ ${{ github.ref }} == 'refs/heads/main' ]; then
echo "::set-output name=runner::ubuntu-latest"
else
echo "::set-output name=runner::macos-latest"
fi
job1:
needs: [setup]
runs-on: ${{ needs.setup.outputs.runner }}
steps:
- run: echo "My runner is ${{ needs.setup.outputs.runner }}" #ubuntu-latest if main branch
I've made a test here if you want to have a look:
workflow file
workflow run
I have a package that is a core dependency of multiple other packages within my organization. My goal is to write an action to automate/ facilitate testing of these reverse dependencies. Roughly, the action should:
Trigger on a comment in a PR.
Run the unit tests of a set of reverse-dependencies with the code in that PR.
Reply to the PR with a comment about which tests failed (if any).
Steps 1 and 3 I got to work, but I’m running into issues with step 2. My current solution is to hardcode all job outputs to pass the results from step 2 to step 3, but I'm wondering if there is a way to avoid hardcoding this.
This following example workflow illustrates my problem:
name: Test
on: push
jobs:
unit-tests:
runs-on: ${{ matrix.os }}
continue-on-error: true
name: ${{ matrix.os }} (${{ matrix.pkg }})
strategy:
fail-fast: false
matrix:
# there will be more pkgs and OSes
os: [ubuntu-latest]
pkg: [pkgA, pkgB]
# how to avoid hardcoding these?
outputs:
ubuntu-latest-pkgA: ${{ steps.update-output.outputs.ubuntu-latest-pkgA }}
ubuntu-latest-pkgB: ${{ steps.update-output.outputs.ubuntu-latest-pkgB }}
steps:
- uses: actions/checkout#v2
- name: fake unit tests
run: |
exit 1 # fail all tests for now
shell: bash
- name: set error if tests fail
id: update-output
if: ${{ failure() }}
run: echo "::set-output name=${{ matrix.os }}-${{ matrix.pkg }}::error"
shell: bash
aggregate-results:
runs-on: ubuntu-latest
needs: unit-tests
steps:
- name: Aggregate results
env:
NEEDS: ${{ toJSON(needs) }}
run: echo "$NEEDS"
The job aggregate-results (inspired by this post) works nicely and prints:
{
"unit-tests": {
"result": "success",
"outputs": {
"ubuntu-latest-pkgA": "error",
"ubuntu-latest-pkgB": "error"
}
}
}
which I can use to create an informative comment. However, the job unit-tests requires me to hardcode the outputs for all combinations of os and pkg. Is there a way to do this dynamically?
I'm looking for this solution as well.
The only way to accomplish this approach is switch to re-usable workflows which allow to access one specific job context
https://docs.github.com/en/actions/learn-github-actions/contexts#jobs-context