I have a reusable workflow for building and pushing a docker image to ECR.
One of the inputs of the workflow is for specifying arguments for the docker build command.
This is the command in the reusable workflow:
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG ${{ inputs.DOCKER_BUILD_ARGS }} .
In some cases, I need DOCKER_BUILD_ARGS to contain secrets, for example:
secrets:
AWS_ACCESS_KEY: ${{ secrets.AWS_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
uses: XXXXX/.github/workflows/DockerBuildPushECR.yml#main
with:
ECR_REGISTRY: XXXXXX
ECR_REPOSITORY: XXXXX
DOCKER_BUILD_ARGS: "--build-arg PASSWORD=${{ secrets.PASSWORD }}"
GitHub complains that the workflow is not valid:
"Unrecognized named-value: 'secrets'", because it only expects secrets in the secrets section.
I cannot pass it as a secret because the reusable workflow does not expect this secret, I just want it to be part of the string...
Can't use env because it cannot be used in conjunction with a reusable workflow
How can I make this scenario work?
What I ended up doing is adding 2 optional secrets to the reusable workflow added them as build args in the docker build commnd.
This way, if they are passed - they are secrets, and if they are not - they are simply blank and this does not affect anything.
It solved my scenario.
So, the secrets section looked like this:
secrets:
AWS_ACCESS_KEY:
required: true
AWS_SECRET_ACCESS_KEY:
required: true
USERNAME:
required: false
PASSWORD:
required: false
and the build like this:
- name: Build and tag image
run: docker build -f ${{ inputs.DOCKERFILE }} -t ${{ inputs.ECR_REGISTRY }}/${{ inputs.ECR_REPOSITORY }}:${{ inputs.IMAGE_TAG }} --build-arg USERNAME=${{ secrets.USERNAME }} --build-arg PASSWORD=${{ secrets.PASSWORD }} ${{ inputs.DOCKER_BUILD_ARGS }} ${{ inputs.DOCKER_BUILD_CONTEXT }}
Of course, the Dockerfile needs to have corresponding arguments.
This allowed me to pass up to 2 secrets "dynamically"
Related
I want to automate the CI process where the tool I use is connected to the GitHub and there are 2 dbs After a developer pushes to one db, the second db should have the capability to pull the resources that were pushed in the first db. The tool (hosted on aws) provides a .sh file which triggers the pull for the second db. How can I connect to the aws instance from GitHub using actions and point to the aws folder and make use of the .sh file to trigger the pull.
I am new to Github and could not find suitable solution to resolve my issue.
Looking for any help/advice. Thanks
A good start would be to use the unfor19/install-aws-cli-action, to benefit from aws CLI.
You can see an example in "The CI / CD pipeline of Github Action for serverless lambda function containerization deployment." from Dr. Tri Basuki Kurniawan.
steps:
- name: Install AWS CLI
uses: unfor19/install-aws-cli-action#v1
with:
version: 1
env:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ ->
secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_DEFAULT_REGION }} - name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ ->
secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_DEFAULT_REGION }} - name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login#v1 - name: Check out code
uses: actions/checkout#v2
...
But depending on the nature of your database, you also have more specialized actions like "Labs: Cross-Region Replication for RDS".
I have a number of GitHub actions that interact with Azure using the az command line, so I figured I'd try to write a reusable workflow to log into Azure. I have been following this guide: https://docs.github.com/en/actions/using-workflows/reusing-workflows
When I run my caller workflow, I get this error:
Error: .../log-into-azure/action.yml (Line: 21, Col: 14): Unrecognized named-value: 'secrets'. Located at position 1 within expression: secrets.DEV_APPLICATION_ID
My caller workflow contains this:
- name: Azure login with elevated permissions
uses: ./.github/actions/log-into-azure
with:
secrets: inherit
My reusable workflow looks like this:
name: Log into Azure
description: 'Log into Azure.'
on:
workflow_call:
secrets:
DEV_APPLICATION_ID:
required: true
DEV_SERVICE_PRINCIPAL_SECRET:
required: true
TENANT_ID:
required: true
jobs:
azure-login:
runs-on: [self-hosted, ubuntu-latest]
steps:
- name: Azure login with elevated permissions
shell: pwsh
run: |
az login --service-principal -u "${{ secrets.DEV_APPLICATION_ID }}" -p "${{ secrets.DEV_SERVICE_PRINCIPAL_SECRET }}" --tenant "${{ secrets.TENANT_ID }}"
I have also tried to list the secrets explicitly in the caller workflow (instead of using secrets: inherit) like this:
- name: Azure login with elevated permissions
uses: ./.github/actions/log-into-azure
with:
secrets:
DEV_APPLICATION_ID: ${{ secrets.DEV_APPLICATION_ID }}
DEV_SERVICE_PRINCIPAL_SECRET: ${{ secrets.DEV_SERVICE_PRINCIPAL_SECRET }}
TENANT_ID: ${{ secrets.TENANT_ID }}
... but that gave the following error message:
The workflow is not valid. .github/workflows/deploy.yml (Line: 60, Col: 11): A mapping was not expected
EDIT 1
I have also tried to put secrets on the same indentation level as uses in my caller workflow, like this (lines 63-65):
- name: Azure login with elevated permissions
uses: ./.github/actions/log-into-azure
secrets: inherit
That also fails:
Invalid workflow file: .github/workflows/deploy.yml#L65
The workflow is not valid. .github/workflows/deploy.yml (Line: 65, Col: 7): Unexpected value 'secrets'
Likewise if I do this:
- name: Azure login with elevated permissions
uses: ./.github/actions/log-into-azure
secrets:
DEV_APPLICATION_ID: ${{ secrets.DEV_APPLICATION_ID }}
DEV_SERVICE_PRINCIPAL_SECRET: ${{ secrets.DEV_SERVICE_PRINCIPAL_SECRET }}
TENANT_ID: ${{ secrets.TENANT_ID }}
I get the exact same error message.
EDIT 2
Here is a minimal working example of my whole caller workflow:
name: Deploy to persistent environment
on:
workflow_dispatch:
jobs:
deploy-kms-to-persistent-environment:
name: 'Deploy KMS to ${{ github.event.inputs.deployment_target}} from Git commit: ${{ github.sha }}'
runs-on: [self-hosted, 3shape-ubuntu-latest]
steps:
- name: Azure login with elevated permissions
uses: ./.github/actions/log-into-azure
secrets: inherit
Checking the official documentation, your problem occurs due to the indentation in the workflow calling the reusable workflow.
You are informing secrets that way:
uses: ...
with:
secrets:
And it should be using secrets at the same level as with:
uses: ...
with:
secrets:
Using your example, both options should look like this:
- uses: ./.github/actions/log-into-azure
secrets: inherit
and
- uses: ./.github/actions/log-into-azure
secrets:
DEV_APPLICATION_ID: ${{ secrets.DEV_APPLICATION_ID }}
DEV_SERVICE_PRINCIPAL_SECRET: ${{ secrets.DEV_SERVICE_PRINCIPAL_SECRET }}
TENANT_ID: ${{ secrets.TENANT_ID }}
Note: In both case, with should be use for inputs, and not for secrets.
Example:
uses: ...
with:
input1: value1
secrets:
secret1: ${{ secrets.SECRET1 }}
Moreover, note that you don't specify the runner and steps when calling a reusable workflows. You just specify the reusable workflow path with the uses field (with the ref), as you already configured the runner and the steps IN the reusable workflow.
In your case, it seems you're calling an action in the workflow, not a reusable workflow.
Example (compare to your workflow in the EDIT 2):
name: Deploy to persistent environment
on:
workflow_dispatch:
jobs:
job1:
uses: owner/repo/.github/workflows/log-into-azure.yml#main #you need the ref here
secrets: inherit
I have created an action that runs the following command
- name: Create Release Notes File
env:
COMMITTEXT: "${{ github.event.head_commit.message }}"
run: |
php ./create-release-notes-file.php
The file is in the same folder as the create-release.yml which is .github/workflows
when the action runs it fails with the following error:
Run php ./create-release-notes-file.php
Could not open input file: ./create-release-notes-file.php
Error: Process completed with exit code 1.
I can't find any documentation that tells me to put the files anywhere else. Any suggestions welcomed
Be sure to have cloned the repo before the above step (like with the actions/checkout#v3), as example:
jobs:
create-release-note:
name: Create Release note
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v3
with:
fetch-depth: 0
persist-credentials: false
token: ${{ secrets.GH_API_TOKEN }}
- name: Create Release Notes File
env:
COMMITTEXT: "${{ github.event.head_commit.message }}"
run: |
php ./create-release-notes-file.php
I have some github workflows using secrets in the repository.
I created the secret, but I used the wrong value for it so I went on the website again and updated the value to a different string.
It seems though that no matter what I do, the value doesn't change when the workflow is running. Outputting the value to the console shows the initial value that I set.
I even tried removing the secret and re-adding it with the new value, no success.
Any idea how to get it to change?
You probably forgot to pass the secrets on in the Actions yml
Changing this did the trick for me.
Adding them in the yaml file as env passes them on to the workflow
jobs:
run:
runs-on: ubuntu-latest
environment: publish
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.9
uses: actions/setup-python#v2
with:
python-version: 3.9
cache: 'pip'
- name: Install dependencies
run: pip install -r requirements.txt
- name: Run
run: python publish.py
env:
URL: ${{ secrets.URL }}
USER: ${{ secrets.USER }}
PASS: ${{ secrets.PASS }}
I need to build docker image form the source code of the current repository, run a container, then execute some API calls. How to do that with the github action?
name: Docs Generator
on:
pull_request:
types: [opened]
jobs:
pr-labeler:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v2
with:
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Get the version
id: vars
run: echo ::set-output name=tag::$(echo ${GITHUB_REF:10})
- name: Build the tagged Docker image
run: docker build . --file Dockerfile --tag service:${{steps.vars.outputs.tag}}
- name: Run docker image
docker run -v ${{ inputs.path }}:/src service:${{steps.vars.outputs.tag}}
- name: Call API
run: |
curl +x http://localhost:8080/test
.....
For this purpose, you could use a combination of https://github.com/marketplace/actions/build-and-push-docker-images and https://github.com/addnab/docker-run-action
The first would build and publish a container, and the second would take this container and run your commands there.
The example is below. I don't use this setup myself but I have tested it. Replace username/container with your username and container.
name: Docker Image CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
compile:
name: Build and run the container
runs-on: ubuntu-latest
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action#v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action#v1
- name: Login to DockerHub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action#v2
with:
push: true
tags: username/container
- name: Check out the repo
uses: actions/checkout#v2
- name: Run the build process with Docker
uses: addnab/docker-run-action#v3
with:
image: username/container:latest
options: -v ${{ github.workspace }}:/data
run: |
echo "Hello World"
Note that building a container is quite a long task and might deplete your Github Action limits quickly. You might consider building/publishing a container separately, or add better caching here (i.e. to rebuild it only on Dockerfile change)
Note that you need to set up DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets.
Instead of echo "Hello World", use the commands you want to run. The repo data will be in the /data directory, for this setup.