I am trying to run a job on multiple os using matrix strategy. my code is below -
strategy:
matrix:
os: [ubuntu-18.04, ubuntu-20.04, self-hosted]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout#v2
However if one of the triggerd job complete with success or failure, all other paralell jobs on remaining os from matrix get terminated automatically.
How can I change this behavior?
I am expecting my all jobs from matrix to keep running till end.
Thanks to answer here - https://stackoverflow.com/a/70841204/4042883
adding below attribute helped -
strategy:
fail-fast: false
Related
Is there a way to use a higher resource runner machine for a specific value in matrix strategy?
this is a sample GitHub action:
test-app:
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
matrix:
run: ['app1', 'app2', 'app3']
for app3 I need 2x resources on the runner machine. Could someone please help me? I searched a lot but could find the answer here on stackoverflow
You need two changes to make this happen:
Define an object to reference your runs-on values
Reference those in the job definition
I have only been able to do this by referencing outputs from a previous job:
jobs:
test-setup:
runs-on: ubuntu-latest
outputs:
runners: '{"app1":"ubuntu-latest", "app2": "ubuntu-latest", "app3": "ubuntu-22.04-16core"}'
steps:
- run: echo no-op
test-app:
needs: [test-setup]
runs-on: ${{ fromJSON(needs.test-setup.outputs.runners)[matrix.run] }}
timeout-minutes: 60
strategy:
matrix:
run: ['app1', 'app2', 'app3']
steps:
- run: echo ${{ matrix.run }}
This is the runs-on value: ${{ fromJSON(needs.test-setup.outputs.runners)[matrix.run] }}, it just pulls the string from the previous job's output
With the matrix strategy and two jobs like those below, the first job runs a "matrix" of possible configurations in parallel, and when all of them succeed, it moves onto the second job.
jobs:
job1:
runs-on: ubuntu-latest
strategy:
matrix:
version: [10, 12, 14]
steps:
# etc
job2:
runs-on: ubuntu-latest
needs: job1
strategy:
matrix:
version: [10, 12, 14]
steps:
# etc
What would be nice is to essentially have a matrix over the whole workflow, where for each matrix value, the entire workflow of jobs is run unimpeded, in parallel. That means for a particular configuration job2 can proceed right after job1 completes for that configuration, regardless of whether job1 for another configuration is still in process or has failed.
In the following diagram, the first case is what will currently happen, and the second is the desired case:
I feel like this could be "emulated" by launching the workflow several times with a different context. But how could this be done, for example from a trigger? Somehow specify on: push that 3 versions of the workflow must run simultaneously?
After some digging, I think "Reusable workflows" is the currently supported method.
The solution is to have two workflow files. One that has all the jobs, and another small workflow that has the matrix and calls the other workflow.
name: Common Workflow
on:
workflow_call:
inputs:
version: # the variable you can use in place of a matrix
required: true
type: number
jobs:
job1:
runs-on: ubuntu-latest
steps:
- run: echo 'job1 version ${{ inputs.version }}'
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo 'job2 version ${{ inputs.version }}'
name: Master Workflow
on:
push:
jobs:
version-matrix:
strategy:
# super important if you want to see all results, even if one fails
# fail-fast is true by default
fail-fast: false
matrix:
version: [10, 12, 14]
uses: ./.github/workflows/common-workflow.yml # calls the one above ^
with:
version: ${{ matrix.version }}
secrets: inherit
Here's a real example of it running in the UI:
One thing you will run into once you run this though, is the UI lists the subjobs in alphabetical order, and not the order they are listed in the workflow. So you lose the sense of the dependencies between the jobs. This is true in February 2023, so maybe Github will improve the UI down the road. Until then, we got around this by naming our jobs like 1. First Job, 2. Second Job so alphabetical is equivalent to the order they run in. It's hacky but it works.
Another thing to note, especially for deployments is the fail-fast attribute. This should be set to false to make sure every workflow in the matrix gets to run to completion, even if one of them fails.
on: pull_request
jobs:
job1:
runs-on: [re-centos7]
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "test=world1" >> $env:GITHUB_OUTPUT
- id: step2
run: echo "test=world" >> $env:GITHUB_OUTPUT
job2:
runs-on: [re-centos7]
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}}
output
I'm trying to use the job's output in the next job. I'm trying to use github output for this but I see it's getting an empty value for the output. Can someone help? Thanks!
output is empty for ${{needs.job1.outputs.output1}}
I'm using self-hosted runners which are integrated with Kubernetes cluster
The first thing to check is the version of your self-hosted runners: The deprecation of set-output was mentioned recently (oct. 2022)
If you are using self-hosted runners make sure they are updated to version 2.297.0 or greater.
If the version is lower, that would explain the empty values.
I'm developing a Github actions workflow. This workflow runs on Linux, Mac, and Windows.
As part of the workflow, I have to check whether 2 environment variables are equal. If they don't - fail the job.
As described here, Github Actions support if: condition:
steps:
- run: # How can I make a cross-platform failure here?
if: ${{ envA }} != ${{ envB }}
How can I make the job fail if the above condition is true?
In the beginning, I thought of a script, but there must be a more elegant way to fail a job.
I'd do run: exit 1. That will simply exit with an exit code of 1, on all three platforms.
Proof that it's cross-platform: https://github.com/rmunn/Testing/runs/220188838 which runs the following workflow:
name: Test exiting on failure
on: [push]
jobs:
build:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout#v1
- name: Try to fail
run: exit 1
- name: Print message if we don't fail
run: echo Should not get here
(An earlier version of this answer recommended "/bin/false", but that would only work on Linux and macOS).
In 2021, there is perhaps a more graceful way to do this:
- name: A/B Check
if: ${{ envA }} != ${{ envB }}
uses: actions/github-script#v3
with:
script: |
core.setFailed('envA and envB are not equivalent!')
Here, we use the github-script action to provide a one liner script that will fail the job. The "A/B Check" step will only run if the condition in the if line is true, so the script will only run in that case, which is what we want.
The nice thing about this approach is that you will get nicely formatted output in the Actions UI in your repo, showing that the "A/B Check" step caused the failure, and why (i.e. "envA and envB are not equivalent").
Note that if you have additional steps in the job after this, and you do NOT want them to run if the A/B check fails, you'll want to use if: success() on them to prevent them from running in that case.
The Github workflow commands docs gives a hint on this.
Toolkit function
Equivalent workflow command
core.setFailed
Used as a shortcut for ::error and exit 1
Given that, you can do the following without using any external workflows.
steps:
- name: A/B Check
if: ${{ envA }} != ${{ envB }}
run: |
echo "::error file={name},line={line},endLine={endLine},title={title}::{message}"
exit 1
When running a GitHub Actions matrix workflow, how can we allow a job to fail, continue running all the other jobs, and also mark the workflow itself as failed?
Below in this image, you can see that the workflow passes even after a job failed. We need to mark the workflow as failed in this case.
Here is the small portion of my workflow yaml file.
continue-on-error line will continue the workflow even if a job fails but how do we get the whole workflow marked as failed?
matrixed:
runs-on: ubuntu-latest
continue-on-error: true
timeout-minutes: 60
defaults:
run:
shell: bash
working-directory: myDir
strategy:
matrix:
testgroups:
[
"bookingpage-docker-hub-parallel",
"bookingpage-docker-hub-parallel-group-1",
"bookingpage-payments",
]
I did find this unanswered question, but this is about steps and we need to know about jobs.
Use fail-fast: false for the strategy and don't set continue-on-error on the job.
matrixed:
runs-on: ubuntu-latest
timeout-minutes: 60
defaults:
run:
shell: bash
working-directory: myDir
strategy:
fail-fast: false
matrix:
testgroups:
[
"bookingpage-docker-hub-parallel",
"bookingpage-docker-hub-parallel-group-1",
"bookingpage-payments",
]