I'm using GitHub Actions and Docker WatchTower to update my images on the fly when I check-in my software (no, it is not crucial software. It is more important to have a lean CI/CD).
name: Docker Image CI
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Build the Docker image
run: docker build . -t me/myrepo:${{github.run_number}}
#:$(date +%s)
#docker build --rm -t ne/myrepo .
- name: Login to Docker Hub
env:
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: |
echo $DOCKER_USER
docker login -u $DOCKER_USER -p $DOCKER_PASSWORD
- name: Push the new Tag to Docker Hub
run: |
docker push me/myrepo:${{github.run_number}}
This works very nicely.
But watch tower can only download the latest version of a version e.g. latest.
Best solution would be I could keep the incrementing versions on github actions and watchtower would take the highest version. I guess it cannot do that.
or -
I tag the latest version (e.g. 49) also as the latest. How would you do that with git hub actions? This should be nothing else than giving multiple tags to a build, no?
Well, I actually answered it almost myself when phrasing the question.
Simply create 2 builds and 2 images. One incrementing (so that you always could roll-back to an older version) and update the latest version.
Prioritize the latest so that it is available faster.
- name: Build the Docker image
run: docker build . -t me/myrepo:latest
- name: Build the Docker image
run: docker build .
-t me/myrepo:${{github.run_number}}
and then push it twice.
- name: Push the also the latest Tag to Docker Hub
run: |
docker push me/myrepo:latest
- name: Push the new Tag to Docker Hub
run: |
docker push me/myrepo:${{github.run_number}}
Related
When I run skaffold in a github workflow like this
skaffold build
it calls the gradle jib correctly, creates an image and pushes it to the ghcr successfully. Grdale finishes successfully as can be seen in the log. Nevertheless, something happens afterwards that fails. It seems someone tries to access the just built image but is not authorized. This does not happen, if I execute it locally. And it does not fail in the github workflow if I call gradlew jib directly without skaffold being involved.
Built and pushed image as ghcr.io/tobias-neubert/motd-service:453f4c4-dirty
BUILD SUCCESSFUL in 11s
4 actionable tasks: 4 executed
time="2023-02-15T12:07:09Z" level=error msg="No matching credentials were found for \"ghcr.io\""
time="2023-02-15T12:07:09Z" level=error msg="No matching credentials were found for \"ghcr.io\""
getting image: GET https://ghcr.io/token?scope=repository%3Atobias-neubert%2Fmotd-service%3Apull&service=ghcr.io: UNAUTHORIZED: authentication required
Error: Process completed with exit code 1.
The github workflow:
name: Build and push motd-service
on:
push:
permissions:
packages: write
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout#v3
- name: Set up Java
uses: actions/setup-java#v2
with:
java-version: 17
distribution: temurin
- name: Setup Gradle
uses: gradle/gradle-build-action#v2
- name: Make gradlew executable
run: chmod +x ./gradlew
- name: Install skaffold
run: |
curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64 && \
sudo install skaffold /usr/local/bin/
- name: Deactivate collecting skaffold metrics
run: skaffold config set --global collect-metrics false
- name: Build the motd image
env:
GH_PASSWORD: '${{ secrets.GITHUB_TOKEN }}'
run: skaffold build
Does anybody know what happens here?
It tries to fetch the digest of the new image, which it needs to render the k8s resources. Pushing the image was made by gradle. The jib plugin is configured to use environment variables for authenticating against ghcr.io. But skaffold does not know about those. So it fails to authenticate. A docker login does the trick, although it is not safe in a CI. So now I have to search for a better way to tell skaffold to authenticate against the registry
I am trying to create a job which has a test app needed for integrated tests. The test app is a simple application which is built and run first and then the final build runs which uses port 8080 on localhost for its integrated tests. However port 8080 is not available when the second docker build runs.
jobs:
my-job:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Build and run testapp
run: >
docker build
-t testapp
-f testapp/Dockerfile .
docker run
-p 8080:80
-d -t testapp
docker build
-t app
-f Dockerfile .
Is this a limitation of github actions? I have started looking at service containers as a workaround, but is there any way that the port can be made available for the subsequent commands in the step?
I have a CodeIgniter Web app connected with Mysql that is developed in the docker. I would like to do some unit test in the GitHub action fo ci/cd pipeline. The problem is some of the function would require enquiry data from Mysql database. So may I know if there is a way to setup a MySQL instance on Github action and run some .sql file so that my test data is in the database?
You can use GitHub Actions service containers to connect to databases like mysql. You can find details at https://docs.github.com/en/actions/guides/about-service-containers
I think this script can help people to conduct unit test with MySQL database and a .sql script file to load table schema and data on Github action. I am using Codeigniter 4 with Mysql. But the process of setting up a database would be similar as long as you are using MySQL.
name: CI Pipeline
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Build the Docker image
run: docker-compose build
- name: up mysql and apache container runs
run: docker-compose up -d
#I am using codeIgniter4
- name: install dependencies
run: docker exec CI4_1 php composer.phar install
- name: buffering time for db container
uses: jakejarvis/wait-action#master
with:
time: '30s'
#db_1 is the name of database
- name: load database
run: docker exec -i mysql_1 mysql -uroot -proot db_1< ./testDatabase.sql
- name: unit test
run: docker exec CI4_1 ./vendor/bin/phpunit
I have an action on master branch which on push/merge builds a package, uploads it to PyPI then checks out to develop branch, bumps version in develop branch and pushes to the origin of develop branch. Develop branch has an action that listens to push/merge and does a snapshot release.
When I push to develop the develop action works perfectly and does a snapshot release, but when master branch pushes, push is successful but the action does not get triggered. What am I missing?
Both actions are added below.
name: Build and Upload Package to PyPI | Master Branch
on:
push:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Set up Python
uses: actions/setup-python#v1
with:
python-version: '3.5'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel twine
pip install GitPython
pip install bumpversion
- name: Strip 'snapshot' from version
run: sed -i 's/-snapshot//g' setup.py
- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
TWINE_REPOSITORY_URL: https://pypi.domain.com
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
- name: Bump Verison and Push to develop
run: |
git stash
git config --local user.email "name#email.com"
git config --local user.name "username"
git checkout develop
python bump_version.py
cat .bumpversion.cfg
git remote set-url --push origin https://username:$GITHUB_TOKEN#github.com/repo/path
git push origin develop
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
name: Build and Upload Package to PyPI | Develop Branch
on:
push:
branches:
- develop
jobs:
bumpTag_build_and_publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Set up Python
uses: actions/setup-python#v1
with:
python-version: '3.5'
- name: Install dependencies for setup
run: |
python -m pip install --upgrade pip
pip install setuptools wheel twine
- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
TWINE_REPOSITORY_URL: https://pypi.domain.co,
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
Provided secrets.GITHUB_TOKEN is intentionally not allowed to trigger workflows. As seen in documention:
(...) if an action pushes code using the repository's GITHUB_TOKEN, a new workflow will not run even when the repository contains a workflow configured to run when push events occur.
If you need your automagic push to be "visible" by workflows, you need to create Personal Access Token, add it to repo secrets, and use that instead of GITHUB_TOKEN.
Note that GitHub assumes that you know what you're doing, if you use non-stock token - which means preventing possible infinite loop is on you. While it's not a case in your scenario for now (develop branch does not push anything), it's worth to remember in case one of workflows will change some day.
I have a github action that runs when a branch is merged into master. It should tag the repo with a version number that it obtains from setup.py, and then push the tag. It should then build the package and upload it to a package repository.
Progress so far: Building and uploading works, tagging does not
name: Deploy Library
on [push]
jobs:
build:
runs-on: ubuntu latest
steps:
- uses: actions/checkout#master
- name: Set up Python env
uses: actions/setup-python#v1
with:
python-version: '3.6'
- name: Install Deps
run: |
python -m pip install --upgrade pip
pip install wheel
pip install twine
- name: Build
run: |
python setup.py build bdist_wheel
- name: Tag
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
VERSION=*sed magic on setup.py*
git tag v$VERSION
git push origin v$VERSION
Everything works except for the git push at the end. The logs complain about the need for a username and password (I only have the GITHUB_TOKEN), and anyway, actions/checkout didn't complain...
I've checked the github actions page, and I can't find one relating to tagging.
The actions/checkout#v1 action leaves the git repository in a detached HEAD state. So in order to push back to the repository there are a few steps required.
Set git config for the user you want to be the commit author:
git config --global user.name 'My User'
git config --global user.email 'myuser#example.com'
Set the remote:
git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}#github.com/username/repository
You may also need to checkout. You can extract the branch name from the GITHUB_REF:
git checkout "${GITHUB_REF:11}"
Related questions and answers:
Push to origin from GitHub action
Unable to commit and push back changes made by github action (invalid user)