I'm trying to utilize some of the default github actions env vars within my composite github action step. I recall reading (the link is long lost) somewhere that composite steps must be passed all of their parameters and don't have access to all the env vars, so I'm trying to do that. However, I can't get the env var value to successfully get passed.
Notice here I'm using 3 different forms of syntax:
- name: Do a thing
uses: ./.github/actions/my-action
with:
repositoryUrl: ${{ env.GITHUB_SERVER_URL }}/$GITHUB_REPOSITORY
commitSha: ${GITHUB_SHA}
context: ${{ env.DOCKER_CONTEXT_PATH }}
tags: ${{ needs.generate-tag.outputs.DOCKER_IMAGE }}
dockerfile: ${{ env.DOCKERFILE_PATH }}
push: true
platforms: linux/amd64, linux/arm64
Those env vars get used by my action, and they're all read as a normal string except for GITHUB_SERVER_URL which is empty. I know this because I can see this snippet in the build output:
--label org.opencontainers.image.source="/$GITHUB_REPOSITORY" --label org.opencontainers.image.revision="${GITHUB_SHA}"
Then within .github/actions/my-action/action.yml I have:
name: "Build image"
description: "Build and conditionally push an image to a remote ECR registry"
inputs:
repositoryUrl:
description: "The github repository URL"
required: true
commitSha:
description: "The commit sha associated with this image"
required: true
tags:
description: "If a new image is built, it will be assigned all of these tags"
required: true
context:
description: "Docker context to use when building"
required: true
dockerfile:
description: "Path to Dockerfile"
required: true
platforms:
description: "Platforms to build"
required: true
push:
description: "Whether to push the image after it's built"
required: true
default: 'false'
runs:
using: "composite"
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action#v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action#v2
- name: Build docker image
uses: docker/build-push-action#v3
with:
context: ${{ inputs.context }}
tags: ${{ inputs.tags }}
file: ${{ inputs.dockerfile }}
platforms: ${{ inputs.platforms }}
push: ${{ inputs.push }}
labels: |
org.opencontainers.image.source="${{ inputs.repositoryUrl }}"
org.opencontainers.image.revision="${{ inputs.commitSha }}"
How can I ensure these values are passed dynamically without having to hard code them as strings?
Looks like in order to utilize github's native env vars within a composite step's with: section, you need to use ${{ github.server_url }} instead of any form of GITHUB_SERVER_URL env var. This means my action looks like:
runs:
using: "composite"
steps:
- name: Generate labels
id: labels
shell: bash
run: echo "CREATED=$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_OUTPUT
- name: Set up QEMU
uses: docker/setup-qemu-action#v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action#v2
- name: Build docker image
uses: docker/build-push-action#v3
with:
context: ${{ inputs.context }}
tags: ${{ inputs.tags }}
file: ${{ inputs.dockerfile }}
platforms: ${{ inputs.platforms }}
push: ${{ inputs.push }}
labels: |
org.opencontainers.image.source="${{ github.server_url }}/${{ github.repository }}"
org.opencontainers.image.revision="${{ github.sha }}"
org.opencontainers.image.created="${{ steps.labels.outputs.CREATED }}"
ref: https://docs.github.com/en/actions/learn-github-actions/contexts#github-context
Related
I am trying to automate some tests but I would need to pass some particular parameters to the final test scripts that would fit perfectly as a json file. The issue now is to make github action able to handle json data as a parameter.
The constraint is that the json file as to be local as the workflow has to be triggered from the command gh workflow run ...
So far I tried create my first yml file as such :
name: setup
on:
workflow_dispatch :
inputs:
config_file:
description: 'json file containing the configuration for the runners'
required: true
type: string
...
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
jobs:
setup-auth:
name: setup-authentication
uses: ./.github/workflows/single-device-authentication.yml
with:
devices: mlops
config_file: ${{ inputs.config_file }}
secrets: inherit
single-device-authentication.yml looks like this, I commented where it fails :
name: single-device-authentication
on:
workflow_call:
inputs:
devices:
required: true
type: string
config_file:
description: 'json file containing the configuration for jetson runners'
required: true
type: string
jobs:
device-authentication:
name: device-authentication
runs-on: ${{ inputs.devices }}
steps:
- uses: PATH/TO/gh_auth#main
with:
app_id: 7
private_key: ${{ secrets.MLOPS_BOT_PRIVATE_KEY }}
json-parser:
name: parser
runs-on: ${{inputs.devices}}
needs: device-authentication
steps:
- name: parser script
run: |
echo ${{ inputs.config_file }}" # This fails
Also, to trigger the workflow, I tried that way :
gh workflow run setup.yml -f config_file="$(cat ${PATH_TO_CONFIG_FILE})"
I have a workflow that uses cache.
The workflow works fine when it triggered on push/manually with 'workflow dispatch'
but when it triggered with 'repository dispatch' meaning it is triggered by another job, I never get a cache hit and all the dependencies are installed from scratch.
This is my workflow:
name: Caching with npm
name: build and trigger release
on:
repository_dispatch:
types: [ release ]
workflow_dispatch:
branches:
- test-branch
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Configure AWS CLI
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Login to Amazon Public ECR
uses: docker/login-action#v1
with:
registry: ****.dkr.ecr.us-east-1.amazonaws.com
username: ${{ secrets.AWS_ACCESS_KEY_ID }}
password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Create ecr repo incase it doesn't exist
uses: int128/create-ecr-repository-action#v1
with:
repository: sharon-test
lifecycle-policy: lifecycle-policy.json
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action#master
- name: Cache Docker layers
uses: actions/cache#v2
with:
path: /tmp/.buildx-sharon-test-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Build Docker image
uses: docker/build-push-action#v2
with:
context: .
builder: ${{ steps.buildx.outputs.name }}
push: true
tags: ****.dkr.ecr.us-east-2.amazonaws.com/sharon-test:latest, ****.dkr.ecr.us-east-2.amazonaws.com/sharon-test:${{ github.event.client_payload.tag }}
cache-from: type=local,src=/tmp/.buildx-sharon-test-cache
cache-to: type=local,dest=/tmp/.buildx-sharon-test-cache-new
file: Dockerfile.api
- name: Move cache
run: |
rm -rf /tmp/.buildx-sharon-test-cache
mv /tmp/.buildx-sharon-test-cache-new /tmp/.buildx-sharon-test-cache
I would also like to mention that In this example you see I am using the local cache
but in the 'Build docker image' part I also used github actions global cache:
- name: Build Docker image
uses: docker/build-push-action#v2
with:
context: .
builder: ${{ steps.buildx.outputs.name }}
push: true
tags: ****.dkr.ecr.us-east-2.amazonaws.com/sharon-test:latest, ****.dkr.ecr.us-east-2.amazonaws.com/sharon-test:${{ github.event.client_payload.tag }}
cache-from: type=gha
cache-to: type=gha,mode=max
file: Dockerfile.api
but it doesn't change a thing,
the cache hit fails in 'Cache Docker layers' evreytime.
Does anyone have a clue of what might be the issue with repository dispatch? or any other issue?
Thank you!
For some reason, the github action I'm working on isn't updating. Whenever I add new arguments to the action, it still uses the old action
Here's the action.yml file: (stored at .github/actions/godot-build/action.yml)
name: Build Godot
description: Build Godot with the provided options.
inputs:
pre:
description: Runs before scons command
default: ""
target:
description: The scons target (debug/release_debug/release).
default: "debug"
tools:
description: If tools are to be built.
default: false
tests:
description: If tests are to be built.
default: false
platform:
description: The Godot platform to build.
required: false
sconsflags:
default: ""
scons-cache:
description: The scons cache path.
default: "${{ github.workspace }}/.scons-cache/"
scons-cache-limit:
description: The scons cache size limit.
# actions/cache has 10 GiB limit, and GitHub runners have a 14 GiB disk.
# Limit to 7 GiB to avoid having the extracted cache fill the disk.
default: 7168
runs:
using: "composite"
steps:
- name: Scons Build
shell: sh
env:
SCONSFLAGS: ${{ inputs.sconsflags }}
SCONS_CACHE: ${{ inputs.scons-cache }}
SCONS_CACHE_LIMIT: ${{ inputs.scons-cache-limit }}
run: |
echo "Building with flags:" ${{ env.SCONSFLAGS }}
${{ inputs.pre }} scons p=${{ inputs.platform }} target=${{ inputs.target }} tools=${{ inputs.tools }} tests=${{ inputs.tests }} ${{ env.SCONSFLAGS }}
ls -l bin/
But when I use it, via a step like this:
- name: Compilation (bits=64)
uses: ./.github/actions/godot-build
with:
pre: "PATH=/home/hp/tmp/x86_64-godot-linux-gnu_sdk-buildroot/bin:$PATH"
sconsflags: ${{ env.SCONSFLAGS }} ${{ env.MONO_SCONSFLAGS }} ${{ matrix.sconsflags }} bits=64 ${{ matrix.build-mono && 'mono_prefix=$HOME/mono-installs/desktop-linux-x86_64-release' || '' }}
platform: linuxbsd
target: ${{ matrix.target }}
tools: ${{ matrix.tools }}
I get the error
Warning: Unexpected input(s) 'pre', valid inputs are ['target', 'tools', 'tests', 'platform', 'sconsflags', 'scons-cache', 'scons-cache-limit']
Is it possible for Github to cache actions? If so, how can I clear this cache?
Turns out I was extracting a tar.gz file into the current directory, which caused it to replace my custom github actions file with it's own github actions file. I fixed this by adding the --exclude=".github" flag to the tar command.
I have this main.yml workflow right here:
name: Testing
on:
push:
branches:
- main
jobs:
upgrade-kubectl:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Install kubectl version
uses: ./.github/actions/promote-image
with:
kubectl: 'latest'
and my action.yml metadata file:
name: "Helm, Kubectl or Devspace installation setup"
description: "Install a specific version of Helm, Kubectl or Devspace. Acceptable values are latest or version strings like 1.15.0"
inputs:
kubectl:
description: "Version of Kubectl"
required: false
helm:
description: "Version of Helm"
required: false
devspace:
description: "Version of Devspace"
required: false
runs:
using: "composite"
steps:
- name: Setting up kubectl
uses: azure/setup-kubectl#v1
with:
version: ${{ inputs.kubectl }}
- name: Setting up Helm
uses: azure/setup-helm#v1
with:
version: ${{ inputs.helm }}
- name: Setting up Devspace
uses: loft-sh/setup-devspace#main
with:
version: ${{ inputs.devspace }}
Currently I am just supplying the kubectl version in my workflow, but when the action is triggered it is running all 3 steps instead.
How do I make it so that if I supply one version it only runs the one step, supply two version it runs two steps respectively, etc.
Any help would be appreciated !
It seems that now conditions are supported on composite actions.
Therefore, you could add if conditions at each step level according to the input used.
In that case, your action.yml workflow file would look like this:
name: "Helm, Kubectl or Devspace installation setup"
description: "Install a specific version of Helm, Kubectl or Devspace. Acceptable values are latest or version strings like 1.15.0"
inputs:
kubectl:
description: "Version of Kubectl"
required: false
helm:
description: "Version of Helm"
required: false
devspace:
description: "Version of Devspace"
required: false
runs:
using: "composite"
steps:
- name: Setting up kubectl
if: ${{ inputs.kubectl != '' }}
uses: azure/setup-kubectl#v1
with:
version: ${{ inputs.kubectl }}
- name: Setting up Helm
if: ${{ inputs.helm != '' }}
uses: azure/setup-helm#v1
with:
version: ${{ inputs.helm }}
- name: Setting up Devspace
if: ${{ inputs.devspace != '' }}
uses: loft-sh/setup-devspace#main
with:
version: ${{ inputs.devspace }}
Note that the syntax:
if: ${{ inputs.kubectl != '' }} works
if: ${{ inputs.kubectl }} != '' doesn't
Let's take this example composite action found on Github's documentation:
name: 'Hello World'
description: 'Greet someone'
inputs:
who-to-greet: # id of input
description: 'Who to greet'
required: true
default: 'World'
outputs:
random-number:
description: "Random number"
value: ${{ steps.random-number-generator.outputs.random-id }}
runs:
using: "composite"
steps:
- run: echo Hello ${{ inputs.who-to-greet }}.
shell: bash
- id: random-number-generator
run: echo "::set-output name=random-id::$(echo $RANDOM)"
shell: bash
- run: ${{ github.action_path }}/goodbye.sh
shell: bash
How can we use that specific output random-number in an external workflow that calls this action? I tried the following snippet but currently it seems the workflow cannot read the output variable from the action as it just comes out empty - 'Output - '
jobs:
test-job:
runs-on: self-hosted
steps:
- name: Call Hello World
id: hello-world
uses: actions/hello-world-action#v1
- name: Comment
if: ${{ github.event_name == 'pull_request' }}
uses: actions/github-script#v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'Output - ${{ steps.hello-world.outputs.random-number.value }}'
})
It seems my attempt was correct with the exception of one detail:
Instead of:
${{ steps.hello-world.outputs.random-number.value }}
It should be referenced without the .value:
${{ steps.hello-world.outputs.random-number}}
Now it works.