I encountered issue that env defined by secret env is empty. I want to use secret env in run syntax. I defined env of secret env like here.
- name: Deploy
env:
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
run: |
date_time=`date +%Y%m%d%H%M%S`
IMAGE=gcr.io/$GCP_PROJECT_ID/web-api-server:$date_time
but $GCP_PROJECT_ID is empty.
invalid argument "gcr.io//web-api-server:20200718163842" for "-t, --tag" flag: invalid reference format
See 'docker build --help'
of course. I confirmed that $GCP_PROJECT_ID is defined at a secret.
The reason is here.
Organization secrets can only be used by public repositories on your plan.
If you would like to use organization secrets in a private repository, you will need to upgrade your plan.
Related
I'm going to setup a workflow on GitHub which uses some docker images to check my package on different platform. I'd like to use the matrix strategy. Then I wrote a workflow like this:
name: Main Workflow
on: [push]
jobs:
test_linux:
runs-on: ubuntu-latest
strategy:
matrix:
image: [ubuntu, fedora]
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Test
uses: docker://${{ matrix.image }}:latest
with:
entrypoint: /bin/bash
args: ./test.sh
This is just a demo workflow. The real one is much more complicated.
However, after I pushed it, GitHub gave me the following error message:
Invalid workflow file: .github/workflows/main.yml#L16
The workflow is not valid. .github/workflows/main.yml (Line: 13, Col: 15): Unrecognized named-value: 'matrix'. Located at position 1 within expression: matrix.image
Actually, I got this error instantly when I pushed codes.
Why? Is the matrix context inaccessible when parsing workflow file? Are there some ways to resolve it?
I tried to export the full docker name to environment in the previous step before "Test", but context env is also inaccessible just like the context matrix. I got nearly the same error. The only different is:
Unrecognized named-value: 'env'...
Now I have to call docker run manually in run scripts like
steps:
- name: Test
run: docker run --rm -v $(pwd):/code --workdir=/code ${{ matrix.image }}:latest /bin/bash -c "/code/test.sh"
It did work but I still prefer to use the uses entry.
I am writing a GitHub Action that does some CD and it uses yq to insert environment variables into a yaml file for deployment.
I'm trying to read a JSON from a GH secret that will eventually be read from env and loaded into python, where said string will be evaluated as a dictionary.
Running this in a terminal, for example:
yq -i '.value="{\"web\": \"test\"}"' test.yaml
Gives me:
value: '{"web": "test"}'
But in a Github Action, where I am doing this:
env:
JSON="{\"web\": \"test\"}"
...
- name: test
run : |
yq -i '
.value=strenv(JSON)
' deployment.yaml
Gives me:
Error: Bad expression, please check expression syntax
Doing other variations of that string, e.g. '{\"web\": \"test\"}', '\"{\"web\": \"test\"}\"' etc also gives me the same error.
I've tried searching on the yq repository and consulted the documentation but can't seem to find what I am looking for.
To summarise, my problem is that I want to read a JSON string as a string when it is evaluated by yq.
One of yq users has recently contributed to yq's github action docs regarding using env variables in github actions - it may help here:
- name: Get an entry with a variable that might contain dots or spaces
id: get_username
uses: mikefarah/yq#master
with:
cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
- name: Reuse a variable obtained in another step
run: echo ${{ steps.get_username.outputs.result }}
See https://mikefarah.gitbook.io/yq/usage/github-action for more info.
Disclaimer: I wrote yq
Here's a GitHub Actions workflow file for a Python project named spam:
name: PyInstaller
on:
[...]
jobs:
create_release:
[...]
make_artifact:
needs: create_release
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
runs-on: ${{ matrix.os }}
env:
ARTIFACT_PATH: dist/spam.zip
ARTIFACT_NAME: spam-${{ runner.os }}.zip
steps:
[...]
When this runs, the workflow fails at startup with this:
The workflow is not valid. [...]:
Unrecognized named-value: 'runner'. Located at position 1 within expression: runner.os
I'm attempting to use the os attribute of the runner context. This SO Q&A mentions that the env context can only be used in specific places, so I suspect something similar is happening here. However, I can't find any official documentation addressing this.
Is there any way to reference the runner context to set an environment variable within the env clause of a job, as shown above?
I'm looking for a way to set the environment variable for all steps in the job, so an env inside a step item won't do.
The workaround I've got for now is to add a step specifically to set environment variables:
steps:
- name: Setup environment
run: |
echo "ARTIFACT_NAME=spam-${{ runner.os }}.zip" >> $GITHUB_ENV
however this only works on the Linux runner.
If you scroll down a bit further in the GitHub Actions docs you linked, there's an example workflow printing different contexts to the log.
- name: Dump runner context
env:
RUNNER_CONTEXT: ${{ toJson(runner) }}
I set up a test repo with a workflow demonstration:
on: push
jobs:
one:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- 'ubuntu-latest'
- 'windows-latest'
- 'macos-latest'
steps:
- name: Dump runner context
env:
RUNNER_CONTEXT: ${{ toJson(runner) }}
run: echo "$RUNNER_CONTEXT"
- name: Get runner OS
env:
RUNNER_OS: ${{ runner.os }}
run: echo "$RUNNER_OS"
- name: Create file with runner OS in name
env:
OS_FILENAME: 'spam-${{ runner.os }}.zip'
run: |
echo "OS_FILENAME=spam-${{ runner.os }}.zip" >> $GITHUB_ENV
touch "./${{ env.OS_FILENAME }}"
touch blah.txt
- name: List created file
run: ls -l "./${{ env.OS_FILENAME }}"
It looks like you can also set and access env in steps, and those persist across workflow steps. For example, I set the environment variable $OS_FILENAME in step 3 using the echo syntax, and reference it in step 4. This works across all the OS options offered on GitHub Actions.
Note that the GitHub Actions docs state that "Environment variables must be explicitly referenced using the env context in expression syntax or through use of the $GITHUB_ENV file directly; environment variables are not implicitly available in shell commands.". Basically, it means you can't implicitly refer to env variables like $FOO and instead must refer to them as ${{ env.FOO }} in shell commands.
So for your scenario, does it satisfy your requirements if you set $ARTIFACT_NAME in the first step of the job? I wonder if the reason might be that the runner context isn't created until the first step - I'm not sure I have a way of testing this.
Github Actions' actions/setup-python step doesn't succeed due to the use of ::set-env: and ::add-path:, that are blocked because considered as insecure (https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands).
Succeed by setting ACTIONS_ALLOW_UNSECURE_COMMANDS=true environment variable for this step.
How to avoid allowing unsecure commands to successfully run a Python build?
Self-hosted runner in a custom-built docker with Ubuntu 20.04 base image. Linked to an GHE server v2.22.4.
Tried with both Python 3.9.0 and 3.9.1
Workflow file:
name: CD testing
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
runs-on: [ Linux ]
- uses: actions/checkout#v2
- name: setup python
uses: actions/setup-python#v2
with:
python-version: 3.9.0
- name: execute py script
run: |
python -V
##[debug]Found tool in cache Python 3.9.1 x64
::set-env name=pythonLocation::/opt/hostedtoolcache/Python/3.9.1/x64
##[error]Unable to process command '::set-env name=pythonLocation::/opt/hostedtoolcache/Python/3.9.1/x64' successfully.
##[error]The `set-env` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
##[debug]System.Exception: The `set-env` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
##[debug] at GitHub.Runner.Worker.SetEnvCommandExtension.ProcessCommand(IExecutionContext context, String line, ActionCommand command, ContainerInfo container)
##[debug] at GitHub.Runner.Worker.ActionCommandManager.TryProcessCommand(IExecutionContext context, String input, ContainerInfo container)
::add-path::/opt/hostedtoolcache/Python/3.9.1/x64
##[error]Unable to process command '::add-path::/opt/hostedtoolcache/Python/3.9.1/x64' successfully.
##[error]The `add-path` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
##[debug]System.Exception: The `add-path` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
##[debug] at GitHub.Runner.Worker.AddPathCommandExtension.ProcessCommand(IExecutionContext context, String line, ActionCommand command, ContainerInfo container)
##[debug] at GitHub.Runner.Worker.ActionCommandManager.TryProcessCommand(IExecutionContext context, String input, ContainerInfo container)
::add-path::/opt/hostedtoolcache/Python/3.9.1/x64/bin
##[error]Unable to process command '::add-path::/opt/hostedtoolcache/Python/3.9.1/x64/bin' successfully.
##[error]The `add-path` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
##[debug]System.Exception: The `add-path` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
##[debug] at GitHub.Runner.Worker.AddPathCommandExtension.ProcessCommand(IExecutionContext context, String line, ActionCommand command, ContainerInfo container)
##[debug] at GitHub.Runner.Worker.ActionCommandManager.TryProcessCommand(IExecutionContext context, String input, ContainerInfo container)
::set-output name=python-version::3.9.1
##[debug]='3.9.1'
Successfully setup CPython (3.9.1)
Update actions/setup-python which uses set-env in previous versions from
- uses: actions/setup-python#v2
to the latest version
- uses: actions/setup-python#v2.2.1
There's a bug - v2 does not use the latest version.
I am new to github actions, and I see two things used for configuring the steps (correct me if i am wrong), with and env.
What is the difference between these two and how they are used.
uses: someAction
with:
x: 10
y: 20
env:
x1: 30
y2: 40
with - is specifically used for passing parameters to the action
env - is used specifically for introducing environment variables that can be accessed depending on scope of the resource
workflow envs - can be accessed by all resources in the workflow except services
job envs - can be accessed by all resources under job except services
step envs - can be accessed by any resource within the step
Here is an example on how parameters are handled
Let's say an action is created with following parameter in action.yaml
name: 'Npm Audit Action'
inputs:
dirPath:
description: 'Directory path of the project to audit'
required: true
default: './'
Then we will provide this parameter through the with tag in our workflow
- name: Use the action
uses: meroware/npm-audit-action#v1.0.2
with:
dirPath: vulnerable-project
Then in the action code we would handle it as such if building a Node.js action
const core = require("#actions/core");
const dirPath = core.getInput("dirPath");
Envs within actions are accessed differently, let's say we are building a Node.js action, then we would access it through process.env. Going back to our example action
name: 'Npm Audit Action'
env:
SOME_ENV: 'hey I am an env'
Then this could be accessed as
const { someEnv: SOME_ENV } = process.env
You can see in the documentation with: used to define a variable.
While env defines an environmnent variable, as defined here and in jobs.<job_id>.env
an environment variable defined in a step will override job and workflow variables with the same name, while the step executes.
A variable defined for a job will override a workflow variable with the same name, while the job executes.
You need both to access secrets:
steps:
- name: Hello world action
with: # Set the secret as an input
super_secret: ${{ secrets.SuperSecret }}
env: # Or as an environment variable
super_secret: ${{ secrets.SuperSecret }}