I created secrets in github actions and trying to use them in reusable workflow, but I am unable to make it work, However, If I pass secrets hardcoded from caller file, it works just fine
## set_env.yml
name: Sent Env Creds and Vars
on:
push:
branches:
- main
- dev
pull_request:
branches: [ main ]
jobs:
deploy-dev:
uses: ./.github/workflows/main.yml
with:
AWS_REGION: "us-east-2"
PREFIX: "dev"
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
reusable workflow = main.yml
## main.yml
name: Deploy to AWS
# Controls when the workflow will run
on:
workflow_call:
inputs:
AWS_REGION:
required: true
type: string
PREFIX:
required: true
type: string
secrets:
AWS_ACCESS_KEY_ID:
required: true
AWS_SECRET_ACCESS_KEY:
required: true
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
terraform-deploy:
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v2
# Runs a set of commands using the runners shell
- name: Run a multi-line script
run: |
echo Hello, Epsilon! You are in ${{ inputs.AWS_REGION }} region ${{ inputs.PREFIX }} region
for dir in $(ls -l | grep '^d' | awk '{print $9}'); do
PARENT_DIR=`pwd`
echo $dir
cd $dir
terraform init -backend-config=${PARENT_DIR}/${{ inputs.PREFIX }}-backend.tfvars
terraform validate
terraform plan -var-file=${{ inputs.PREFIX }}_vars.tfvars
## terraform apply -input=false -auto-approve -var-file=${{ inputs.PREFIX }}_vars.tfvars
cd ..
done
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
If I hardcode secrets in set_env.yml while calling main.yml like below, it just works
jobs:
deploy-dev:
uses: ./.github/workflows/main.yml
with:
AWS_REGION: "us-east-2"
PREFIX: "dev"
secrets:
AWS_ACCESS_KEY_ID: <harcoded value>
AWS_SECRET_ACCESS_KEY: <hardcoded value>
I have been trying to make it work in many ways but doesnt work. Please help
As of May 3rd 2022, this is now possible with the new keyword inherit: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onworkflow_callsecretsinherit
In the calling workflow, you tell it to inherit the secrets in the reusable workflow:
jobs:
deploy-dev:
uses: ./.github/workflows/main.yml
with:
AWS_REGION: "us-east-2"
PREFIX: "dev"
secrets: inherit
This makes the secrets available in the reusable workflow like normal:
with:
myInput: ${{ secrets.MY_SECRET }}
Note that there's no need to declare the secrets on the workflow_call trigger.
I was running into this issue. For me the culprit was the secret value in Github secrets. The secret had been created correctly, it had the correct value and name however Github actions could not find it for some reason. Deleting the secret and recreating it seems to have solved the issue though i cannot determine why
Related
I am struggling with the recent change on how GitHub Action Workflows defines runtime variables, replacing the "set-ouput" approach with environment variables.
Last summer, it took me a couple of hours to figure out the code below working for my requirement. Define a matrix of OS and python versions, so that the CI workflow can create corresponding environments and run the pytests on it.
Is there any chance to get support on how to best transform to the new approach?
env:
# JSON variables (used in our strategy/matrix)
SUPPORTED_PYTHON_VERSIONS: '\"python-version\":[\"3.8\", \"3.9\"]'
SUPPORTED_OPERATING_SYSTEMS: '\"os\":[\"ubuntu-latest\", \"macos-latest\", \"windows-latest\"]'
jobs:
# The set-env job translates the json variables to a usable format for the workflow specifications.
set-env:
runs-on: ubuntu-latest
outputs:
py-os-matrix: ${{ steps.set-matrix-vars.outputs.py-os-matrix }}
# ^ this represents:
# matrix:
# - os: [ubuntu-latest, ...]
# - python-version: [3.7, ...]
os-matrix: ${{ steps.set-matrix-vars.outputs.os-matrix }}
# ^ this represents:
# matrix:
# - os: [ubuntu-latest, ...]
steps:
- id: set-matrix-vars
run: |
echo "::set-output name=py-os-matrix::{${{ env.SUPPORTED_PYTHON_VERSIONS }},${{ env.SUPPORTED_OPERATING_SYSTEMS }}}"
echo "::set-output name=os-matrix::{${{ env.SUPPORTED_OPERATING_SYSTEMS }}}"
test-run:
name: test on ${{ matrix.os }} - ${{ matrix.python-version }}
needs: set-env
strategy:
fail-fast: true
matrix: ${{ fromJson(needs.set-env.outputs.py-os-matrix) }}
Meanwhile, a more elegant way of using the matrix keyword functions is available. My new implementation is much cleaner and, most important, avoids the above-mentioned deprecated function of printing env variables to stdout. However, the point is open how to integrate python versions >=3.10 ...
jobs:
test-run:
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
python-version: [3.8, 3.9]
defaults:
run:
shell: bash
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository code
uses: actions/checkout#v3
- name: Install python
id: setup-python
uses: actions/setup-python#v4
with:
python-version: ${{ matrix.python-version }}
check-latest: true
Every further step which comes after the one with setup-python is executed in the currenlty selected combination of OS and python.
I want to use environment variables at the job level. Is there a way to do it?
env:
stageEnv: UAT
jobs:
name: Upload Build
if: ${{ env.stageEnv == 'UAT' }}
steps:
....
I get unrecognized named-value: 'env' error. Tried $stageEnv and ${{ env.stageEnv }}
Note: It works when I access within 'steps', but would like this to be accessible at 'jobs' level.
I'm afraid not, but you can do like this:
env:
stageEnv: UAT
jobs:
build:
name: Build
runs-on: ubuntu-latest
outputs:
stageEnv: ${{ steps.init.outputs.stageEnv }}
steps:
- name: Make environment variables global
id: init
run: |
echo "stageEnv=${{ env.stageEnv }}" >> $GITHUB_OUTPUT
And use it in another job:
upload:
name: Upload build
needs: build
if: ${{ needs.build.outputs.stageEnv == 'UAT' }}
Note this is just an example, and I personally prefer environment variables uppercase and output variables lowercase
We have two runners, one for running production jobs and another for running non production jobs, but I am unable to do that using a workflow level environment variable.
Below is what I have:
name: Workflow file
on:
workflow-dispatch
env:
RUNNER_NAME: ${{ contains(github.ref, 'main') && 'Prod Runner' || 'non-Prod Runner' }}
jobs:
job-run:
runs-on: [${{ env.RUNNER_NAME }}]
needs: ...
steps:
..........
I get the following error message:
Invalid workflow file
You have an error in your yaml syntax on line ###
How do I do this? I don't want to have separate workflow files for prod and non-prod workflows.
For what you can check on this Github Actions ISSUE, it seems it's not possible to use natively env variable on the runs-on job field (yet?).
However, there is a workaround if you configure a variable as output in a previous job, so you would be able to use it afterwards.
Example: runs-on: ${{ needs.setup.outputs.runner }}
In your case, the workflow would look like this:
on:
workflow_dispatch:
jobs:
setup:
runs-on: ubuntu-latest
outputs:
runner: ${{ steps.step1.outputs.runner }}
steps:
- name: Check branch
id: step1
run: |
if [ ${{ github.ref }} == 'refs/heads/main' ]; then
echo "::set-output name=runner::ubuntu-latest"
else
echo "::set-output name=runner::macos-latest"
fi
job1:
needs: [setup]
runs-on: ${{ needs.setup.outputs.runner }}
steps:
- run: echo "My runner is ${{ needs.setup.outputs.runner }}" #ubuntu-latest if main branch
I've made a test here if you want to have a look:
workflow file
workflow run
I have a package that is a core dependency of multiple other packages within my organization. My goal is to write an action to automate/ facilitate testing of these reverse dependencies. Roughly, the action should:
Trigger on a comment in a PR.
Run the unit tests of a set of reverse-dependencies with the code in that PR.
Reply to the PR with a comment about which tests failed (if any).
Steps 1 and 3 I got to work, but I’m running into issues with step 2. My current solution is to hardcode all job outputs to pass the results from step 2 to step 3, but I'm wondering if there is a way to avoid hardcoding this.
This following example workflow illustrates my problem:
name: Test
on: push
jobs:
unit-tests:
runs-on: ${{ matrix.os }}
continue-on-error: true
name: ${{ matrix.os }} (${{ matrix.pkg }})
strategy:
fail-fast: false
matrix:
# there will be more pkgs and OSes
os: [ubuntu-latest]
pkg: [pkgA, pkgB]
# how to avoid hardcoding these?
outputs:
ubuntu-latest-pkgA: ${{ steps.update-output.outputs.ubuntu-latest-pkgA }}
ubuntu-latest-pkgB: ${{ steps.update-output.outputs.ubuntu-latest-pkgB }}
steps:
- uses: actions/checkout#v2
- name: fake unit tests
run: |
exit 1 # fail all tests for now
shell: bash
- name: set error if tests fail
id: update-output
if: ${{ failure() }}
run: echo "::set-output name=${{ matrix.os }}-${{ matrix.pkg }}::error"
shell: bash
aggregate-results:
runs-on: ubuntu-latest
needs: unit-tests
steps:
- name: Aggregate results
env:
NEEDS: ${{ toJSON(needs) }}
run: echo "$NEEDS"
The job aggregate-results (inspired by this post) works nicely and prints:
{
"unit-tests": {
"result": "success",
"outputs": {
"ubuntu-latest-pkgA": "error",
"ubuntu-latest-pkgB": "error"
}
}
}
which I can use to create an informative comment. However, the job unit-tests requires me to hardcode the outputs for all combinations of os and pkg. Is there a way to do this dynamically?
I'm looking for this solution as well.
The only way to accomplish this approach is switch to re-usable workflows which allow to access one specific job context
https://docs.github.com/en/actions/learn-github-actions/contexts#jobs-context
I know that I can use Runner's Linux environment variables in GitHub Actions.
Do I have any other options to have variables and use them in workflow steps?
This is how variables are designed to work in GitHub Actions. I mean declared variables are mapped to env variables, like here:
name: Show env
on:
push:
branches:
- '*'
env:
somevar: 'lastvar'
jobs:
show:
runs-on: ubuntu-latest
steps:
- name: Is variable exported?
run: |
echo "${{ env.somevar }}"
However, you can't use them everywhere - please check this topic:
env:
pluginId: 'plugin-fn-xml-node'
on:
push:
paths:
- ${{env.pluginId}}/**
- .github/workflows/**
pull_request:
paths:
- ${{env.pluginId}}/**
- '.github/workflows/**'
jobs:
build:
env:
working-directory: ${{env.pluginId}}
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [8.x, 10.x, 12.x]
steps:
This will not work because you can't use workflow variable at job level.
So if you define variable at workflow level you should be able to use it across steps.
I added also dynamically set variable based on documentation
env:
somevar: 'lastvar'
jobs:
show:
runs-on: ubuntu-latest
steps:
- name: Is variable exported?
run: |
echo "${{ env.somevar }}"
echo "action_state=yellow" >> $GITHUB_ENV
- name: PowerShell script
# You may pin to the exact commit or the version.
# uses: Amadevus/pwsh-script#25a636480c7bc678a60bbf4e3e5ac03aca6cf2cd
uses: Amadevus/pwsh-script#v2.0.0
continue-on-error: true
with:
# PowerShell script to execute in Actions-hydrated context
script: |
Write-Host $env:somevar
Write-Host $env:action_state
- name: Read exported variable
run: |
echo "$action_state"
echo "${{ env.action_state }}"