I have set GitHub actions to run some jobs in parallel. I configured the jobs before, but for performance, I started running those jobs in parallel.
This is my pnpm-caching.yaml workflow:
name: Reusable pnpm caching workflow
on:
workflow_call:
inputs:
runner:
required: true
type: string
node-version:
required: true
type: string
pnpm-version:
required: true
type: string
jobs:
install_dependencies:
name: Install dependencies and caching
runs-on: ${{ inputs.runner }}
strategy:
matrix:
node-version: ['${{ inputs.node-version }}']
pnpm-version: ['${{ inputs.pnpm-version }}']
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile
And these are my jobs running upon pull request:
name: Integration
on:
pull_request:
branches: [main]
jobs:
call_pnpm_caching_workflow:
name: Set pnpm caching
uses: ./.github/workflows/pnpm-caching.yaml
with:
runner: ubuntu-latest
node-version: 18
pnpm-version: 7.14.0
test_depcheck:
needs: call_pnpm_caching_workflow
name: Testing Depcheck
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test Depcheck
run: pnpm depcheck && pnpm -r --parallel depcheck
test_inflint:
needs: call_pnpm_caching_workflow
name: Testing Inflint
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test Inflint
run: pnpm inflint
test_typescript:
needs: call_pnpm_caching_workflow
name: Testing TypeScript
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test TypeScript
run: pnpm type-check && pnpm -r --parallel type-check
test_prettier:
needs: call_pnpm_caching_workflow
name: Testing Prettier
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test Prettier
run: pnpm prettier
test_eslint:
needs: call_pnpm_caching_workflow
name: Testing ESLint
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test ESLint
run: pnpm lint && pnpm -r --parallel lint
test_stylelint:
needs: call_pnpm_caching_workflow
name: Testing Stylelint
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test Stylelint
run: pnpm --filter frontend stylelint
test_prisma:
needs: call_pnpm_caching_workflow
name: Testing Prisma
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test Prisma
run: pnpm prisma-format
test_build:
needs: call_pnpm_caching_workflow
name: Testing build
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18]
pnpm-version: [7.14.0]
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Setup Node.js
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
- name: Installing dependencies
run: pnpm i --frozen-lockfile --offline
- name: Test build
run: pnpm -r --parallel build
As you can see, all (but 1) jobs running in parallel.
Before, I had all these job running in sequence and all successfully finished.
But, once I started running these jobs in parallel, I got random errors on random jobs (sometime these errors show up, sometimes not). By that I mean that sometimes the action as a whole runs successfully, but sometimes some random jobs just fail for random errors like:
apps/cli-backend postinstall: Bus error (core dumped) apps/cli-backend postinstall: ELIFECYCLE Command failed with exit code 135. apps/cli-backend postinstall: Failed or apps/cli-backend postinstall: Error: ENOENT: no such file or directory, stat '/home/runner/work/dashboard/dashboard/node_modules/.pnpm/prisma#4.5.0/node_modules/prisma/libquery_engine-debian-openssl-1.1.x.so.node'
All the errors occur in the pnpm i step.
I'm not sure even why. If I re-run the job it successfully runs.
Related
The Error from GitHub:
An error occurred trying to start process '/usr/bin/bash' with working directory '/home/runner/work/myproject-api/myproject-api/app'. No such file or directory
My Workflow File:
name: Docker Build and Push to Docker Hub and ghcr.io
on:
push:
branches:
- 'feature/auto-deploy-dev'
defaults:
run:
working-directory: app
jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Set up QEMU
uses: docker/setup-qemu-action#v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action#v1
- name: Login to DockerHub
uses: docker/login-action#v1
with:
username: ${{ secrets.SECRET }}
password: ${{ secrets.TOKEN }}
- name: Build and push
uses: docker/build-push-action#v2
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
file: Dockerfile
tags: |
repo/project:latest
- name: Checkout
uses: actions/checkout#v2
- name: KubeCtl Command
uses: tale/kubectl-action#v1
with:
base64-kube-config: ${{ secrets.KUBECONFIG }}
- run: kubectl get pods -n myNamespace
I'm trying to integrate Lighthouse CI into my CI/CD to generate reports on my applications performance. I'm using GitHub Actions, and other jobs like building the app and generating a SonarCloud scan are working.
However Lighthouse CI is not working. The error is: Error: fatal: could not read Username for 'https://github.com': terminal prompts disabled
Code:
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
name: Node.js CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
sonarcloud:
name: SonarCloud
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarCloud Scan
uses: SonarSource/sonarcloud-github-action#master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
lighthouse:
name: Lighthouse CI
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v3
with:
token: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
submodules: recursive
- name: Use Node.js 16.x
uses: actions/setup-node#v3
with:
node-version: 16.x
- name: Run the Lighthouse CI
run: |
npm install -g #lhci/cli#0.6.x
lhci autorun
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
matrix:
node-version: [ 16.x ]
steps:
- uses: actions/checkout#v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
- run: npm run build --if-present
- run: npm test
- name: Log in to the Container registry
uses: docker/login-action#f054a8b539a109f9f41c372932f1ae047eff08c9
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action#98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=sha
- name: Build and push Docker image
uses: docker/build-push-action#ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
Screenshot of GitHub Actions:
hi，i find a issue for the problem。
https://github.com/actions/checkout/issues/664
“For a simple checkout indeed no PAT is required.”
so you can try remove token
lighthouse:
name: Lighthouse CI
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v3
with:
# token: ${{ secrets.LHCI_GITHUB_APP_TOKEN }} // remove
submodules: recursive
- name: Use Node.js 16.x
uses: actions/setup-node#v3
with:
node-version: 16.x
- name: Run the Lighthouse CI
run: |
npm install -g #lhci/cli#0.6.x
lhci autorun
Can Github Action cache be used to speedup globally installed node tools?
I am using semantic-release on a ruby repository and I don't want to pollute that repository with package.json
My configuration for semantic-release resides in .releaserc
I can run this GitAction to update my SemVer.
name: SemVer
on:
push:
branches: [ main ]
jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '16'
- name: Install Global Dependencies
run: npm -g install semantic-release #semantic-release/changelog #semantic-release/commit-analyzer #semantic-release/exec #semantic-release/git #semantic-release/release-notes-generator
- name: Run SemVer
run: semantic-release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
It takes 22 seconds to install dependencies
I have tried integrating cache configuration that I found into my workflow, but this is is not working and I assume it is because there is no package-lock.json to build a hash against.
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
name: SemVer
on:
push:
branches: [ main ]
jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '16'
- name: Cache node modules
uses: actions/cache#v2
id: cache-node-modules
env:
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Install Global Dependencies
if: steps.cache.outputs.cache-hit != 'true'
run: npm -g install semantic-release #semantic-release/changelog #semantic-release/commit-analyzer #semantic-release/exec #semantic-release/git #semantic-release/release-notes-generator
- name: Run SemVer
run: semantic-release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
You don't need to hash a package-lock.json if you don't have one, you can hash any other file or either none, eg:
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}
documentation is available here https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
I have read through the documentation for Github actions and the release action, yet I can find nothing on how to get the name from the action. Do I need to run a fetch call or something else?
name: Publish to Bintray
on:
release:
types: [published]
workflow_dispatch:
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: 1.8
- name: Grant execute permission for gradlew
run: chmod +x gradlew
- name: Build with Gradle
run: ./gradlew curseforge uploadSubProjects publishToModrinth --parallel --stacktrace
env:
BINTRAY_USER: oroarmor
BINTRAY_KEY: ${{ secrets.BINTRAY_KEY }}
CURSE_API_KEY: ${{ secrets.CURSE_API_KEY }}
MODRINTH_TOKEN: ${{ secrets.MODRINTH_TOKEN }}
You can use github.event.release.name and github.event.release.body:
name: After Release
on:
release:
types: [published]
jobs:
after-release:
runs-on: ubuntu-latest
steps:
- run: echo "Name: ${{ github.event.release.name }} Description: ${{ github.event.release.body }}"
Is it possible for Github actions to upload a build artifact for commits on a release branch, and then for another workflow to download & use that artifact?
name: Deploy release to UAT & archive artifact
on:
release:
types: [published]
jobs:
package:
name: package and archive
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
with:
node-version: '12'
- name: serverless package
uses: serverless/github-action#master
with:
args: package --stage=prod
- name: Upload Release Asset
uses: actions/upload-release-asset#v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ github.event.release.upload_url }}
asset_path: .serverless
asset_name: release-asset-${{ github.event.release.name }}.zip
asset_content_type: application/zip
- name: Upload Artifact
uses: actions/upload-artifact#v2
with:
name: release-artifact-${{ github.event.release.name }}
path: .serverless
...but how do you download the asset/artifact? I think up/download-artifact is intended only to be used only from the same workflow, and there doesn't seem to be an action for downloading a release asset.
name: Deploy to production
on:
workflow_dispatch:
inputs:
release:
description: Name of release to deploy
required: true
default: v1.0.0
jobs:
deploy:
name: deploy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
with:
node-version: '12'
- run: npm ci --only=prod
- name: Download the release artifact
uses: actions/download-artifact#v2
with:
name: release-${{ github.event.inputs.release }}
path: .serverless
- name: serverless deploy
uses: serverless/github-action#master
with:
args: deploy --stage=prod --package=.serverless
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
You can use download-workflow-artifact action.
One of my team mates got this working:
- id: download-release-asset
name: Download release asset
uses: dsaltares/fetch-gh-release-asset#master
with:
version: tags/${{ env.RELEASE }}
file: myproject-${{ env.RELEASE }}.tar.gz
target: release.tar.gz
token: ${{ secrets.DEPLOY_TOKEN }}