Hello I'm a little confused if it is possible via Github Actions to get the latest SHA of a file with only its file's name.
# This is a basic workflow to help you get started with Actions
name: CI
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the master branch
push:
branches: [ master ]
pull_request:
branches: [ master ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v2
with:
fetch-depth: 0
- name: Get specific changed files
id: changed-files-specific
uses: tj-actions/changed-files#v15.1
with:
files: |
*.groovy
files_ignore: |
*.yml
# Runs a set of commands using the runners shell
- name: echo changed files
run: |
echo modified files ---
echo ${{steps.changed-files-specific.outputs.modified_files}}
As you can see with the combination of action changed-files-specific and echo changed files I am able to get the filename. I looked at the documentation of the tj-actions/changed-files library and it does not provide file info support.
Is there an easy way to do this? I tried searching for another action library but it does not seem to be a very common use case.
Many Thanks,
Morgan Morningstar
You are on the right track.
Now when you have all the modified files and their paths - you can just easily do whatever you want with those files.
You can iterate over those files and calculate SHA for each of them using those paths.
Something like this:
for file in ${{ steps. changed-files-specific.outputs.modified_files }}; do
sha=`sha1sum $file | cut -d ' ' -f 1`
echo "sha for $file: $sha"
done
Related
I've got a workflow in github actions that automatically creates build artifacts and updates a single release with these new build artifacts every time I merge a PR into main (here's the repo).
I want to know if a new PR will cause a change in the build artifacts (specifically, there's just one CSV file that I care about). Sometimes these changes will be intentional, sometimes not, so I want something like a git diff between the CSV file before the PR and the CSV file after the PR.
I know I could setup a github action to:
checkout the old version of the code.
Run the code to generate the build artifacts
save the files of interest to disc
checkout the proposed version of the code from the PR
Run the PR code to generate the build artifacts
git diff the version before the PR to the version after the PR.
Format and write the git diff output as a comment to the PR, letting
me know about what changes there were so I can check that everything's ok manually.
But this seems like a really common problem and I can't believe there's not a simple
tool/solution out there already? Maybe some github action where you give it two SHAs, a command to run, and a list of files to git diff.
To be clear, these are build artifacts, so aren't tracked by git, and so solutions like git diff pullrequest main -- myfile.csv won't work.
Here is a solution that leverages git notes:
(In a nutshell, git notes allow you to CRUD metadata to a commit without touching the commit itself — and thus preserving history. Cf. § References below.)
Essentially, we want our workflow to:
Build the artefactsWe emulate this by running make build — to be adapted to your own scenario. For the sake of the example, we also assume that the build/ directory contains all and only the artefacts generated.
“Remember” the artefacts and their content (a so-called “artefacts summary”)We use the sha512sum shell command to create a mapping of artefacts' content (represented through their SHA sum) to their file name.We retrieve all artefacts via find results/ -type f, and then convert the mapping to a CSV with headers using sed 's/ /,/' | cat <(echo 'sha512,file_name') -
Attach the artefacts summary to the commitThat's where we leverage git notes, which allows us to add metadata to the commit ex-post, without modifying the history.
These steps should be executed for any commit on your main branch.
In case of a PR, you also want to repeat these steps on the branch's HEAD, plus:
Retrieve the artefacts summary of your PR's target branchSo you now have two artefacts summaries to compare: base (your main/master branch's one) and head (the branch of your PR). In the example below, the base is hard coded to main, but you could refine this by letting the workflow retrieve the target branch's name automatically.
Compare both artefacts summariesI've created the artefactscomparison Python package for that purpose. (Note: it's very much tailored to my use case and desiderata.)
Add the artefact comparison report to your PRBeebop, a bot will do that for you.
In the end, you should see something like on the screenshot above.
name: Artefacts Comparison
on:
push:
branches:
- main
pull_request:
branches:
permissions: write-all
jobs:
build_artefacts:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v3
with:
fetch-depth: 0
token: ${{ github.token }}
- name: Build artefacts
run: make build
- name: Generate artefacts summary
id: artefacts-summary
run: |
echo "ARTEFACTS_SUMMARY<<EOF" >> $GITHUB_OUTPUT
find build/ -type f -exec sha512sum {} \; | sed 's/ /,/' | cat <(echo 'sha512,file_name') - >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Add the artefacts summary as a git notes
run: |
git fetch origin refs/notes/*:refs/notes/*
git config user.name "github-actions"
git config user.email "bot#github.com"
git notes add -m "${{ steps.artefacts-summary.outputs.ARTEFACTS_SUMMARY }}"
git notes show
git push origin refs/notes/*
# In case of PR, add report of artefacts comparison
compare_artefacts:
runs-on: ubuntu-latest
if: ${{ github.event_name == 'pull_request' }}
steps:
- name: Checkout
uses: actions/checkout#v3
with:
fetch-depth: 0
token: ${{ github.token }}
- name: Pull artefacts summaries (i.e., git notes) from upstream
run: |
git fetch origin refs/notes/*:refs/notes/*
- name: Retrieve PR's head branch's artefacts summary
id: artefact-summary-head
run: |
echo "ARTEFACTS_SUMMARY<<EOF" >> $GITHUB_OUTPUT
git notes show >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Retrieve PR's target branch's artefacts summary
id: artefact-summary-base
run: |
git checkout ${{ github.base_ref }}
echo "ARTEFACTS_SUMMARY<<EOF" >> $GITHUB_OUTPUT
git notes show >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Setup Python
uses: actions/setup-python#v4
with:
python-version: "3.10"
- name: Install artefactscomparison package
run: pip install -U artefactscomparison
- name: Generate artefact comparison report
id: artefact-comparison-report
run: |
echo "${{ steps.artefact-summary-head.outputs.ARTEFACTS_SUMMARY }}" > head.csv
echo "${{ steps.artefact-summary-base.outputs.ARTEFACTS_SUMMARY }}" > base.csv
echo "ARTEFACTS_REPORT<<EOF" >> $GITHUB_OUTPUT
artefacts_comparison -b base.csv -h head.csv >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Comment PR with artefact comparison report
uses: thollander/actions-comment-pull-request#v2
with:
message: ${{ steps.artefact-comparison-report.outputs.ARTEFACTS_REPORT }}
comment_tag: artefact_comparison_report
mode: recreate
needs: build_artefacts
References:
git notes documentation
How to sync (i.e. “pull” and push) git notes with upstream
git notes | Enhance Git Commit Messages with Notes
Git Notes: git's coolest, most unloved feature
Automatically add git notes via Github Actions
I currently have a Github Actions workflow setup with the following trigger:
on:
pull_request:
paths:
- 'myFolder/*.yml'
I want this workflow to run on pull request events where a file matching with myfolder/*.yml has been changed. While this workflow does run on pull request events where this file has changed, it also runs on subsequent events even if they do not make any further changes.
The workflow this trigger is for runs a process using configuration from within the yml files and so if no changes happen to any of these files between commits (even if other files that do not match the filter are changed), the result will always be the same and does not need to be run.
I looked through the documentation for Github Actions and could not find anything that exactly matches my situation so would appreciate some help and pointers.
A simplified version with some name changes of the full workflow yml is:
name: Read yml files
on:
pull_request:
paths:
- 'myFolder/*.yml'
jobs:
promote:
runs-on: ubuntu-latest
name: Read files
steps:
- uses: actions/checkout#v2
with:
fetch-depth: 0
- name: Read
run: npm run read-yml
It's not possible to do on YML workflow level.
You can however detect your case and leave early from a workflow.
I can suggest using changed-files action:
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files#v17.2
- name: List all added files
run: |
for file in ${{ steps.changed-files.outputs.added_files }}; do
echo "$file was added"
done
or you can use renamed_files, deleted_files if it fits better to your needs.
Then you can detect if there are any files that may trigger your generation action - if not, simply end the workflow.
I'm trying to keep multiple github actions in the same monorepo using subdirectories, and run them like:
workflow.yml
// [...]
jobs:
run_my_script:
runs-on: ubuntu-latest
steps:
- name: Check out current repo
uses: actions/checkout#v2
- uses: ./my_action2
with:
my_input_var: "david"
./my_action2/action.yml
// [...]
runs:
using: "composite"
steps:
# Checkout files in this repo
- name: Checkout
uses: actions/checkout#v1
- name: Run myscript
run: python myscript.py "${{ inputs.my_input_var }}" # location: ./my_action2/myscript.py
shell: bash
The problem I'm having is that my action uses a python script in it's subdirectory, but the uses: action appears to run from the GITHUB_WORKING_DIR of the workflow and not the directory of the action itself.
python: can't open file 'myscript.py': [Errno 2] No such file or directory
I've looked through most of the working-directory questions surrounding github actions, but I'm still stumped.
I've also tried adding working-directory: ./my_action2 to the job's defaults: but it looks like it's not propagating to run: commands within the uses: step.
My workaround in the meantime has been to add an input for myaction2_working_directory in the workflow, and then add working-directory: ${{ inputs.myaction2_working_directory }} to every run: command in the action. This seems inelegant and repetitive. Is there a better way to do this?
contrary to the answer by Grzegorz, you cannot just run: cd foo and then expect all following steps to have a working directory of foo. as far as i can tell, the only way to do this is with the "workaround" the OP already posted -- add an input named e.g. working-directory to your action and then add working-directory: ${{ inputs.working-directory }} to every step. see https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runsstepsworking-directory
I had similar problem and for my composite actions I just added a first step as:
run: cd ${{ inputs.working_directory }}
and then all next steps are running in it.
I couldn't find a better way and having working-directory copy pasted was also something I didn't like.
I currently have a GitHub Action that triggers on:
pull_request_review:
types: [submitted]
I then want to run a command, which expects the contents of changes of the Pull Request.
Previously, I was using
on:
push
and I had no issues with the contents of the files being available in the Action context.
However, my command is failing now, and I think it's because the context only includes the commit that the action was triggered on (no file changes.)
Previously I was running this action on push and that was always successful, with the file changes being available in the context.
I'm using:
steps:
- uses: actions/checkout#v2
(https://github.com/actions/checkout)
Is it possible to use this to have all the file changes on the Pull Request within the Action context?
Any help on this would be appreciated!
You can do that by using an open source Action available on marketplace:
jobs:
build:
runs-on: ubuntu-latest # windows-latest | macos-latest
name: Test changed-files
steps:
- uses: actions/checkout#v2
with:
fetch-depth: 0 # OR "2" -> To retrieve the preceding commit.
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files#v14.6
- name: List all changed files
run: |
for file in ${{ steps.changed-files.outputs.all_changed_files }}; do
echo "$file was changed"
done
The solution above uses git checkout and git diff to get files changed by PR. Alternatively if you really need just information about paths changed and you don't really need files themselves (no checkout) - you can do it without checkout using gh CLI:
gh pr view XXX --json files -q '.files[].path'
You can run it like this:
jobs:
comment:
runs-on: ubuntu-latest
steps:
- run: gh pr view XXX --json files -q '.files[].path'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
After reading this answer:this
I tried to do the same.
I have a .net core project and in my case, I am using a repo with a publish version so my appsettings.json is in the root of the repo.
# This is a basic workflow to help you get started with Actions
name: DeployToStaging
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
pull_request:
types: [assigned, opened, synchronize, reopened]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
FTP-Deploy-Action:
name: FTP-Deploy-Action
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2.1.0
with:
fetch-depth: 2
- uses: microsoft/variable-substitution#v1
with:
files: '${{env.DOTNET_ROOT}}/appsettings.json'
env:
ConnectionStrings.ToBudget: 'This is just a test'
- name: FTP Deploy
uses: SamKirkland/FTP-Deploy-Action#3.1.0
with:
ftp-server: <MyServer>
# FTP account username
ftp-username: <MyUsername>
ftp-password: ${{ secrets.FtpPassword }}
So basically I want to transform my connection string (for now it is just a test, in the future I will create a secret) and then push it to the server through FTP.
Everything is working except the variable substitution. The error is: No file matched with specific pattern: /appsettings.json
Any help would be much appreciated
Just found the issue.
instead of files: '${{env.DOTNET_ROOT}}/appsettings.json' I just need to do files: 'appsettings.json'
Now I am having a second issue. SamKirkland/FTP-Deploy-Action#3.1.0 doesn't like the change. It is avoiding uploading because the repo is dirty.
EDIT: regarding the second issue I moved to sebastionpopp/ftpaction