Only run black on changed files - github-actions

I'm using the official black GitHub Action. Currently, whenever I push changes, black runs on the entire repository. However, I only want it to run on the changed files. I've tried using some of the GitHub environment variables, but to no avail. Here's my workflow yaml:
name: Lint
on: [push, pull_request]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- run : echo ${{ github.sha }} # this outputs a SHA
- run : echo ${{ github.run_attempt }} # this outputs an int
- run: echo ${{ github.head_ref }} # outputs nothing
- run: echo ${{ github.base_ref }} # outputs nothing
- uses: actions/setup-python#v3
with:
python-version: '3.9.12'
name: Run black on diffed files
- run: echo ${{ github.head_ref }} # outputs nothing
- run: echo ${{ github.base_ref }} # outputs nothing
- run: pip install black && black $(git diff --name-only ${{ github.base_ref}} ${{ github.head_ref }} | grep .py)
The workflow successfully installs and runs black, but it fails because no files are being passed to the black command.
I'm not sure what I'm doing wrong here.

Found a solution that works for me, which also takes into consideration the advice of #jonrsharpe (see his comments on the original question).
Utilized another GitHub action to get all changed files. Then, I run black and reorder-python-imports only on the changed files. That way, the user knows which of their files have issues. However, for CI reasons (thanks Jon!), I also then run black on the full codebase using Black's official GHA.
NOTE: At present, the following GHA works as desired on pull-requests. However, if you push multiple commits, it only runs the action on the final commit in the group. I'm trying to figure out how to make it run on all commits that are pushed, rather than just the final one.
name: Black (python)
on: [push, pull_request]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout branch
uses: actions/checkout#v3
with:
fetch-depth: 0
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files#v35
with:
files: **/*.py
- name: Setup Python env
uses: actions/setup-python#v3
- name: Install black and reorder-python-imports
run: pip install black reorder-python-imports
- name: Black and Sort changed files
run: |
black --check reorder-python-imports ${{ steps.changed-files.outputs.all_changed_files }}
- name: Run black on full codebase
uses: psf/black#stable

This solution works for me at the moment - hope it matches what you need.
Basically, got from git the name of the changed files vis-a-vis master, and run black on them. Added some conditions and filters to only run it on non-deleted files and only python files:
changed_files=$(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py)
if [ ! -z "$changed_files" ];
then black -S --line-length 120 $(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py);
fi
And if you want to just check, add the relevant flag:
changed_files=$(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py)
if [ ! -z "$changed_files" ];
then black --check -S --line-length 120 $(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py);
fi

Related

How to run a Github Action Task if merging to master or the VERSION file contains letter b

I would like to publish a Python package to pypi if merging to master OR a file named VERSION contains letter b. The VERSION file is located in the root of this repo.
I'm able to get the "merging to master" part work with the following code.
publish:
needs: [build]
runs-on: [self-hosted, generic-linux]
container: python:3
steps:
- name: Download artifacts
uses: actions/download-artifact#v2
with:
name: package
path: ./dist
- name: Install requirements
run: |
pip install --upgrade pip
pip install --upgrade --prefer-binary twine
- name: Upload to artifactory
if: ${{ github.ref == 'refs/heads/master' }}
env:
TWINE_REPOSITORY_URL: https://artifactory.example.com/artifactory/api/pypi/pypi-all
TWINE_REPOSITORY: pypi-all
TWINE_USERNAME: "${{ secrets.PUBLISH_USERNAME }}"
TWINE_PASSWORD: "${{ secrets.PUBLISH_BEARER_TOKEN }}"
run: |
twine upload --skip-existing --verbose dist/*
However, I'm not sure how to add an OR condition to check the content of a file. Could someone help?
Thanks.
you could add an extra step to read the content (manually or using some existing GH action like https://github.com/marketplace/actions/read-files-action) of the file and add a condition the the Upload step, to check if the file contains the required string (with https://docs.github.com/en/actions/learn-github-actions/expressions#contains), like:
- name: Checkout code
uses: actions/checkout#v3
- name: Read Version
id: version
uses: komorebitech/read-files-action#v1.5
with:
files: '["VERSION"]'
- name: Echo Version
run: echo "${{ steps.version.outputs.content }}"
- name: Upload to artifactory
if: ${{ github.ref == 'refs/heads/master' || contains(steps.version.outputs.content, 'b')}}
Remember to checkout the code of the repo before try to read the file

Informing Sentry about the latest release version in GitHub

I am using the getsentry/action-release#v1 GitHub Action to inform Sentry about new releases in my GitHub application. However, since I am using tags as the version number, I would like to inform sentry about the latest tag available on the release page. I'm having issues doing that while using environment variables.
Here is my job:
inform_sentry_about_release:
runs-on: ubuntu-latest
env:
ACCESS_TOKEN: ${{ secrets.GH_PAT }}
steps:
- uses: actions/checkout#v2
- name: Set GITHUB_VERSION variable
run: |
echo 'GITHUB_LATEST_RELEASE=$(curl -H "Authorization: token ${ACCESS_TOKEN}" "https://api.github.com/repos/myusername/myreponame/releases" -s | jq -r ".[0].tag_name")' >> $GITHUB_ENV
- name: Create Sentry release
uses: getsentry/action-release#v1
env:
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
SENTRY_ORG: ${{ secrets.SENTRY_ORG }}
SENTRY_PROJECT: ${{ secrets.SENTRY_PROJECT }}
with:
environment: production
version: ${{ GITHUB_LATEST_RELEASE }}
My Github workflow is failing with the following error:
The workflow is not valid. .github/workflows/production-label.yml (Line: 131, Col: 20): Unrecognized named-value: 'GITHUB_LATEST_RELEASE'. Located at position 1 within expression: GITHUB_LATEST_RELEASE
According to How do I dynamically set an environment variable in a github composite action step?, it appears that I should be able to just echo a key=val to the GITHUB_ENV and then call it in the next step, but no luck.
Line 131 specifically refers to this:
version: ${{ GITHUB_LATEST_RELEASE }}
Is there a different way I'm supposed to access this environment variable from this line? I've tried $GITHUB_LATEST_RELEASE and still no luck with that either.
The environment variable should be accessed in the following way:
version: ${{ env.GITHUB_LATEST_RELEASE }}
Please note, that the names of environment variables are case-sensitive, and you can include punctuation.
Example:
steps:
- name: Set the value
id: step_one
run: |
echo "action_state=yellow" >> $GITHUB_ENV
- name: Use the value
id: step_two
run: |
echo "${{ env.action_state }}" # This will output 'yellow'
For more information check the Setting an environment variable.

How do I change the #yarn:registry in github action when using actions/setup-node

I am trying to install GSAP 3 with Shockingly Green package.
The following are the steps recommended by the plugin.
//npm.greensock.com/:_authToken=XXXXXXXXXXXXXXXXXXXXXX
#gsap:registry=https://npm.greensock.com
And
yarn add gsap#npm:#gsap/shockingly
This is my workflow file which I am using in github action.
name: Deployment
on:
push:
branches: [ development ]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1.4.4
with:
node-version: 14.16.0
- name: Setup PHP with intl
uses: shivammathur/setup-php#v2
with:
php-version: '7.4'
extensions: intl-67.1
- name: Install Composer
run: sudo composer
- name: Install dependencies
run: |
composer install -o
yarn
env:
NPM_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}
- name: Build
run: yarn build
- name: Sync
env:
dest: 'root#XXXXXXXXXXX:/var/www/html/wp-content/themes/XXXXXXX'
run: |
echo "${{secrets.DEPLOY_KEY}}" > deploy_key
chmod 600 ./deploy_key
rsync -chav --delete \
-e 'ssh -i ./deploy_key -o StrictHostKeyChecking=no' \
--exclude /deploy_key \
--exclude /.git/ \
--exclude /.github/ \
--exclude /node_modules/ \
./ ${{env.dest}}
I have added proper secret NPM_AUTH_TOKEN with the code provided to me by GSAP.
But I keep getting this error.
yarn install v1.22.17
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
error An unexpected error occurred: "https://npm.greensock.com/#gsap%2fshockingly/-/shockingly-3.8.0.tgz: Request failed \"403 Forbidden\"".
info If you think this is a bug, please open a bug report with the information provided in "/home/runner/work/lacadives-theme/lacadives-theme/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Error: Process completed with exit code 1.
I think action is running on wrong registry.
Is there a way that I can change the registry to #gsap:registry=https://npm.greensock.com.
There were two issues which needed to be addressed.
Setting up .npmrc correctly
And removing yarn.lock file
My final yml file.
name: Deployment
on:
push:
branches: [ development ]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1.4.4
with:
node-version: 14.16.0
- name: Create NPMRC
run: |
echo "//npm.greensock.com/:_authToken=XXXXXXXXXXXXXXXXXXXXXXXXXXXX" >> ~/.npmrc
echo "#gsap:registry=https://npm.greensock.com" >> ~/.npmrc
- name: Setup PHP with intl
uses: shivammathur/setup-php#v2
with:
php-version: '7.4'
extensions: intl-67.1
- name: Install Composer
run: sudo composer
- name: Install dependencies
run: |
composer install -o
rm yarn.lock
yarn
- name: Build
run: yarn build
- name: Sync
env:
dest: 'root#XXXXXXX:/var/www/html/wp-content/themes/XXXXXXXXX'
run: |
echo "${{secrets.DEPLOY_KEY}}" > deploy_key
chmod 600 ./deploy_key
rsync -chav --delete \
-e 'ssh -i ./deploy_key -o StrictHostKeyChecking=no' \
--exclude /deploy_key \
--exclude /.git/ \
--exclude /.github/ \
--exclude /node_modules/ \
./ ${{env.dest}}

GitHub action to run command and add commit if I type comment

My objective is to get to the point where I can type /run-black as a comment on a pull request in GitHub, and then GitHubActions will run black . on the pull request's branch and add a commit.
The use case is that sometimes casual contributors make a small pull request to my library (e.g. fixing a typo), and I'd like to be able to just write a comment like /run-black to have the black formatter run on their files before I merge.
Use the action Slash Command Dispatch. Add a repo scoped PAT with the name PAT to your secrets and create two workflows with the following definitions.
name: Slash Command Dispatch
on:
issue_comment:
types: [created]
jobs:
slashCommandDispatch:
runs-on: ubuntu-latest
steps:
- name: Slash Command Dispatch
uses: peter-evans/slash-command-dispatch#v2
with:
token: ${{ secrets.PAT }}
issue-type: pull-request
commands: |
run-black
on:
repository_dispatch:
types: [run-black-command]
jobs:
runBlack:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
with:
repository: ${{github.event.client_payload.pull_request.head.repo.full_name}}
ref: ${{github.event.client_payload.pull_request.head.ref}}
token: ${{ secrets.PAT }}
- name: Slash Command Dispatch
run: black .
- run: |
git config --local user.email "41898282+github-actions[bot]#users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git commit -m "Run black" -a
git push

How to Extract Branch Name on Delete Event Github Actions

I am trying to extract the branch name on a delete event. Turns out it's not in the GITHUB_REF object as that will be the default branch.
Ordinarily I would run
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
id: extract_branch
But apparently with delete events I need to extract the branch via ${{ github.event.ref }}
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${{ github.event.ref }})" # how to drop refs/heads/?
id: extract_branch
now I don't know how to drop the refs/heads aspect of the branch name.
EDIT: Since in the delete event case, github.event.ref already contains the simple branch name e.g. feature-1-my-branch and not refs/heads/feature-1-my-branch my example code above works.
In the event I want to do some post-processing on this context in a different event type, where github.event.ref returns refs/heads/feature-1-my-branch how would I drop the refs/heads in that case?
You can just use ${{ github.event.ref }} to reference the branch name, the full delete event payload is documented in the GitHub API docs.
I also did a test myself. With workflow defined in here.
steps:
- uses: actions/checkout#v2
- name: run build
run: |
echo "GITHUB_SHA is ${{ github.sha }}"
echo "GITHUB_REF is ${{ github.ref }}"
echo "${{ github.event.ref }} - ${{ github.event.ref_type }}"
I can trigger a run by pushing and deleting a branch (it would also apply for tag as well). It leads to a run like this.
GITHUB_SHA is feb56d132c8142995b8fea6fd67bdd914e5e0d68
GITHUB_REF is refs/heads/master
so-62779643-test-delete-event-test2 - branch
[update]
For strip out the prefix in GITHUB_REF, here is what I did:
- uses: actions/checkout#v2
- name: run build
run: |
echo "::set-env name=GITHUB_REF::${{ github.ref }}"
echo "old GITHUB_REF is $GITHUB_REF"
GITHUB_REF=$(echo $GITHUB_REF | sed -e "s#refs/heads/##g")
echo "new GITHUB_REF is $GITHUB_REF"
run log reference
old GITHUB_REF is refs/heads/master
new GITHUB_REF is master