I have a github action pipeline with multiple jobs, and each job has its own checkout step. But I am not sure if this can have some side effects.
What happens when there is another commit when my pipeline is already running? Will my pipeline run then checkout the changes from the other commit?
How can I avoid this? Should I use "actions/cache" instead of checkout the code in every step? Or is "actions/upload-artifact" and "actions/download-artifacts" better? Or maybe a totally different solution?
Example of my pipeline
name: my_job
on: [push]
jobs:
build_and_test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-java#3
with:
distribution: 'adopt'
java-version: '17'
cache: 'maven'
- name: test
run: ./mvnw clean verify
- name: build
run: ./mvnw package
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: infrastructure
run: ./script/infrastructure.sh
- name: deploy app
run: ./script/delploy_app.sh
Related
I have created a github action workflow to deploy code but now i need to add the approval functionality before deploying, once build is done approver should get a mail notification to approve the deployment, how can i achieve this?
below is my workflow file :
name: DEV Workflow
on:
push:
branches: [ dev ]
pull_request:
branches: [ dev ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: '1.8'
distribution: 'adopt'
cache: maven
- name: Build with Maven
run: mvn clean install
- name: Unit Testing
run: echo "Hello World!"
- name: Deploy
run: echo "Hi World!"
Deploy-dev:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: '1.8'
distribution: 'adopt'
cache: maven
- name: Build with Maven
run: mvn clean install
- name: Deploy
run: echo "Hi World!"
unit-testing:
needs: Deploy-dev
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: '1.8'
distribution: 'adopt'
cache: maven
- name: Unit Testing
run: echo "Hello World!"
On Github Actions you can use Environments to set required approvers (this will send emails to the users when needed).
You can list multiple teams / people that can approve the job, and only 1 person needs to approve it from that list for the workflow to continue, as explained to this other section of the documentation:
Use required reviewers to require a specific person or team to approve workflow jobs that reference the environment. You can list up to six users or teams as reviewers. The reviewers must have at least read access to the repository. Only one of the required reviewers needs to approve the job for it to proceed.
If you don't want to use this solution, a workaround could be to use this manual-approval action, which also allows you to inform more than 1 approver.
I have a demo repo https://github.com/sh977218/Angular-with-Playwright to try to run playwright with Angular on GitHub's action and publish the result to GitHub Page. I'm able to achieve that, but I'd like to improve it by publish each Pull Request's playwright result to GitHub Page and they are all available at same time.
Can someone guide me on how to do this?
This is the github action yml file
name: Playwright Tests
on:
push:
branches: [main, master]
pull_request:
branches: [main, master]
# Grant GITHUB_TOKEN the permissions required to make a Pages deployment
jobs:
Playwright-tests:
timeout-minutes: 60
runs-on: ubuntu-latest
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# Grant GITHUB_TOKEN the permissions required to make a Pages deployment
permissions:
actions: write
checks: write
contents: write
deployments: write
id-token: write
issues: write
discussions: write
packages: write
pages: write
pull-requests: write
repository-projects: write
security-events: write
statuses: write
# Deploy to the github-pages environment
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- uses: actions/checkout#v3
- uses: actions/setup-node#v3
with:
node-version: 16
- name: Install dependencies
run: npm ci
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npx playwright test
- uses: actions/upload-artifact#v3
if: always()
with:
name: playwright-report
path: playwright-report/
retention-days: 30
- name: Setup Pages
uses: actions/configure-pages#v2
- name: Upload artifact
uses: actions/upload-pages-artifact#v1.0.7
with:
path: playwright-report/
- name: Deploy Playwright result to Github Pages
id: deployment
uses: actions/deploy-pages#v1.2.4
Thanks!
Giving the following sample Github actions workflow
name: My workflow
on: pull_request
jobs:
foo:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout#v3
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go#v3
with:
go-version: 1.19
- name: Foo
run: echo "foo"
bar:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout#v3
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go#v3
with:
go-version: 1.19
- name: Bar
run: echo "bar"
I want the jobs Foo and Bar to run in parallel. But as you can see they have some steps in common.
Is it possible to create a job that runs the checkout and setup step and provides itself to Foo and Bar so they only have to run their own commands? ( that would save some time, but I don't think that's possible because both jobs run in separate containers )
If that's not possible, is there a way to extract the "duplicate" lines and move them to a "step function" I can call in my jobs so I don't have to write those steps over and over again?
You may use caching to save some time: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows and to consolidate the "duplicated lines in your YAML file" in every job, you may want to have a composite action, where you basically extract
steps:
- name: Checkout repository
uses: actions/checkout#v3
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go#v3
with:
go-version: 1.19
To a new action that you may use in your workflow file to look like:
name: My workflow
on: pull_request
jobs:
foo:
runs-on: ubuntu-latest
steps:
- name: My composite action
uses: path/to/action
- name: Foo
run: echo "foo"
bar:
runs-on: ubuntu-latest
steps:
- name: My composite action
uses: path/to/action
- name: Bar
run: echo "bar"
Note that if you want to create this composite action in the same repository you will have to use actions/checkout#v3 before you call it using a relative URL.
So it will be:
name: My workflow
on: pull_request
jobs:
foo:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: My composite action
uses: ./.github/actions/my-action.yaml
- name: Foo
run: echo "foo"
bar:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: My composite action
uses: ./.github/actions/my-action.yaml
- name: Bar
run: echo "bar"
And yes, if you only have only a few steps this approach may not bring much value to you. As you may only save few lines in the YAML file, and you can only cache your dependencies installation.
And, this doesn't mean that your "shared/composite" actions will run only once, Github will re-run each of their steps for each job that's calling them (foo, bar in your case).
One other approach, to consolidate some of the steps you run in your pipelines is by creating a Docker image in which your actions will run, this base docker image may have the necessary setup for you, for example: GoLang, and your necessary build and test modules installation.
name: My workflow
on: pull_request
jobs:
foo:
runs-on: ubuntu-latest
container: mydocker.image.uri/name:version
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Foo
run: echo "foo"
bar:
runs-on: ubuntu-latest
container: mydocker.image.uri/name:version
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Bar
run: echo "bar"
The gains with this approach are that you may cut some lines out of your workflow file, and you extracted some setup steps to the base docker image, in which you will run your actions.
About the downsides, it might be a little bit challenging to build a stable base image with the necessary setups to run your steps. And also, you will need to maintain another part of your CI/CD pipelines.
One other solution would be the usage of an execution matrix (with Caching dependency files and build outputs), which will run parallel jobs for each of your matrix values (they will be parallelized depending on the runner's availability or by your max-parallel value)
name: My workflow
on: pull_request
jobs:
foo:
runs-on: ubuntu-latest
strategy:
matrix:
greeting: ["hello", "bonjour"]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Setup Go
uses: actions/setup-go#v3
with:
go-version: 1.19
cache: true
- name: Saying ${{ matrix.greeting }}
run: echo "${{ matrix.greeting }}!"
And as you see, all steps re-run again:
which might not be interesting if you are doing more than GoLang setup but doing steps that may take a lot of time
And the last option I have in my mind is to use dependant jobs, which may not work for this use case. But it might be a solution if you can redesign your workflow to produce an output, or a binary from a first step called baz then your workflow would have
foo:
runs-on: ubuntu-latest
needs: baz
steps:
- name: Something
run: echo "baz is saying: ${{ needs.baz.outputs.greeting }}"
I hope this helps or gave you more ideas on how to optimize this workflow further!
I would like to run automated Playwright tests with Github Actions. I created a cronjob for this, but it has no effect. What else should I consider? Here is the content of my .github/workflows/playwright.yml:
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
schedule:
- cron: '12 14 * * Mon-Fri'
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '14.x'
- name: Install dependencies
run: npm ci
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npx playwright test
- uses: actions/upload-artifact#v2
if: always()
with:
name: playwright-report
path: playwright-report/
retention-days: 30
In the future, if you have to create cron jobs, I can recommend crontab.guru. Its UI is simple to create effective cron schedules.
The run came 2 hours late. Patience is required here.
I am new to writing GH Actions. I am working on a task to remove common workflows and use the reusable workflow feature available. I am now able to get my workflows to run sequentially which is great. However, the 2nd workflow is resulting in an unexpected error message seemingly related to the yarn dependency workflow not saving to the cache as I would have expected it:
Run yarn lint
... snip
myPackage: /bin/sh: 1: concurrently: not found
Could you take a look at see if this looks ok? For now, my goal is to have a workflow for pull-request which calls yarn and lint as the resuable features:
name: pull-request
on:
pull_request:
branches:
- main
jobs:
yarn:
uses: ./.github/workflows/yarn.yml
validate_lint:
needs: yarn
uses: ./.github/workflows/validate_lint.yml
with:
name: Yarn
on:
workflow_call:
jobs:
yarn_and_deps:
name: Run Lint
runs-on: ubuntu-latest
steps:
- name: Checkout Git repository
uses: actions/checkout#v3
- name: Enable node
uses: actions/setup-node#v3
with:
node-version: 16
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "::set-output name=dir::$(yarn cache dir)"
- uses: actions/cache#v3
id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
with:
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install dependencies
run: yarn install --frozen-lockfile && yarn bootstrap
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
and
name: Validate Lint
on:
workflow_call:
jobs:
run_lint:
name: Run Lint
runs-on: ubuntu-latest
- name: Enable node
uses: actions/setup-node#v3
with:
node-version: 16
cache: 'yarn' # <<--- THIS CACHE IS NOT FOUND 🤷🏻♂️
# NOTE: if I add in all the "yarn cache/install" commands from above workflow, this passes.
steps:
- name: Validate Lint
run: yarn lint
The error happens here in the Validate Lint job because it appears that the cache is not found. I made the yarn job to avoid re-creating the wheel for each job.
What is wrong with my expectations on the cache v how it actually works? Having to Install dependencies step each job feels like overkill.
Turns out each workflow is its own docker container. Therefore, if I run yarn in workflow 1's container, workflow 2 has no knowledge/access to the cache.
The closest thing appears to be an upload/download "sharing of data", but this has it's own drawbacks - such as downloading a node_modules folder can be slower than just installing the dependencies.
Unfortunately, the solution seems to be that there is repetition of code when each workflow has a dependency to the output of prior item.