I am using a Github Actions pipeline and starting a mysql server using docker within that pipeline. I am attemtping to run a show tables command, however the output is being suppressed.
In the below pipeline i have a docker-compose file which runs a mysql server. Then i am attempting to connect to it and output the tables. However i never receive the output.
pipeline.yml
name: test-pipeline
on: [ push ]
jobs:
test:
name: Test Migration
runs-on: self-hosted
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
with:
node-version: '12'
- name: Setup
run: |
docker-compose -f CI/docker-compose.yml up -d
npm install
- name: Check tables
run: |
mysql --host=127.0.0.1 --protocol=tcp --user=root --password=password testDB -e "show tables;"
echo "here"
If anyone else comes across this problem i fixed it by not specifying the db upfront in the options, instead specifying the database in the -e command.
mysql --host=127.0.0.1 --protocol=tcp --user=root --password=password -e "show databases; use testDB; show tables;"
Related
I'm currently building application and want to test on GitHub actions.
I'm using mysql for database and need to insert test data(csv format) into GitHub action's container, but it's not working well.
Here is the steps.
Create mysql container. → OK
Run script to create table. → NG.
Insert test data into table created on step2. → NG.
In every step, there are no errors, but it seems I can't run scripts in step2 and step3.
Please tell me which part of my code is wrong.
Codes here.
ci.yml
name: sample-ci
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
job-with-mysql-8_0:
runs-on: ubuntu-latest
services:
db:
image: mysql:8
ports:
- 3306:3306
env:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: sampleDB
options: >-
--health-cmd "mysqladmin ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: setup-go
uses: actions/setup-go#v3
with:
go-version: 1.16
- name: checkout
uses: actions/checkout#v3
- name: Show Docker containers
run: docker ps -a
- name: Show databases for root user
run: mysql --protocol=tcp -h localhost -P 3306 -u root -ppassword -e "SHOW DATABASES"
- name: Set up MySQL
run: sudo /lib/systemd/systemd-sysv-install enable mysql
sudo systemctl enable mysql.service
sudo systemctl start mysql.service
- name: Run sample.sql
run: |
mysql --protocol=tcp -h localhost -P 3306 -u root -ppassword -e "$(cat $(find ./ -name sample.sql))"
- name: Show created tables
run: mysql --protocol=tcp -h localhost -P 3306 -u root -ppassword -e "USE sampleDB"
mysql --protocol=tcp -h localhost -P 3306 -u root -ppassword -e "SHOW TABLES"
- name: Insert data into user table
run: sudo /lib/systemd/systemd-sysv-install enable mysql
sudo systemctl enable mysql.service
sudo systemctl start mysql.service
sh ./.github/scripts/sample.sh
- name: Check inserted data
run: mysql --protocol=tcp -h localhost -P 3306 -u root -ppassword -e "USE sampleDB"
mysql --protocol=tcp -h localhost -P 3306 -u root -ppassword -e "SELECT * FROM user"
sample.sql
USE sampleDB;
CREATE TABLE user(
id INT,
name VARCHAR(100) NOT NULL,
PRIMARY KEY(id),
UNIQUE(name)
);
sample.sh
mysql -uroot -ppassword --local-infile=1 sampleDB -e "LOAD DATA LOCAL INFILE '/docker-entrypoint-initdb.d/user.csv' INTO TABLE user FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES"
user.csv
id,name
1,aaa
2,bbb
3,ccc
I'm having issues using mysql as the database for my tests that are running in a Github action. I'm using this as a guide.
I'm getting the following error:
SQLSTATE[HY000] [1045] Access denied for user 'root'#'172.18.0.2' (using password: NO) (SQL: SHOW FULL TABLES WHERE table_type = 'BASE TABLE')
here is my yaml file:
name: LaravelTest
on:
push:
branches: [ test ]
jobs:
laravel_tests:
runs-on: ubuntu-latest
container:
image: kirschbaumdevelopment/laravel-test-runner:8.1
services:
testdb:
image: mysql:5.7
env:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: test
MYSQL_ALLOW_EMPTY_PASSWORD: 1
ports:
- 33306:3306
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
steps:
- uses: actions/checkout#main
- name: Copy .env
run: php -r "file_exists('.env') || copy('.env.example', '.env');"
- name: Install Dependencies
run: composer install -q --no-ansi --no-interaction --no-scripts --no-progress
- name: Generate key
run: php artisan key:generate
- name: Directory Permissions
run: chmod -R 777 storage bootstrap/cache
- name: Install dependencies
run: npm install
- name: Compile assets
run: npm run dev
- name: Execute tests (Unit and Feature tests) via PHPUnit
run: vendor/bin/phpunit
forge_deploy:
runs-on: ubuntu-latest
needs: laravel_tests
steps:
- name: Make Get Request
uses: satak/webrequest-action#master
with:
url: ${{ secrets.MOMENTUM_TEST_DEPLOY_URL }}
method: GET
UPDATE
I removed this line from my phpunit.xml file:
<env name="DB_HOST" value="testdb"/>
and now I'm getting a different error:
SQLSTATE[HY000] [2002] Connection refused (SQL: SHOW FULL TABLES WHERE table_type = 'BASE TABLE')
After doing some more googling, I found this post which made me go back and pull out the docker container and try to do it using the mysql service that's already available on ubuntu.
Once I was able to successfully start the service and create a database, I then realized that the steps in my action were copying the .env file, and if it wasn't there, it was copying the .env.example file, which I overlooked, and was getting odd results. Once I realized that it was looking in that file for database connection values, I was able to override them with the env option in the yaml file. So, I finally got it working, and this is my working yaml file for anyone that may run into this at some point:
name: LaravelTest
on:
push:
branches: [ test ]
jobs:
laravel_tests:
runs-on: ubuntu-20.04
env:
DB_CONNECTION: mysql
DB_HOST: localhost
DB_PORT: 3306
DB_DATABASE: testdb
DB_USERNAME: root
DB_PASSWORD: root
steps:
- name: Set up MySQL
run: |
sudo systemctl start mysql
mysql -e 'CREATE DATABASE testdb;' -uroot -proot
mysql -e 'SHOW DATABASES;' -uroot -proot
- uses: actions/checkout#main
- name: Copy .env
run: php -r "file_exists('.env') || copy('.env.example', '.env');"
- name: Install Dependencies
run: composer install -q --no-ansi --no-interaction --no-scripts --no-progress
- name: Generate key
run: php artisan key:generate
- name: Directory Permissions
run: chmod -R 777 storage bootstrap/cache
- name: Install dependencies
run: npm install
- name: Compile assets
run: npm run dev
- name: Execute tests (Unit and Feature tests) via PHPUnit
run: vendor/bin/phpunit
forge_deploy:
runs-on: ubuntu-20.04
needs: laravel_tests
steps:
- name: Make Get Request
uses: satak/webrequest-action#master
with:
url: ${{ secrets.MOMENTUM_TEST_DEPLOY_URL }}
method: GET
I have a Job that connects to a running MySQL database and executes some commands:
apiVersion: batch/v1
kind: Job
metadata:
name: do-db-stuff
spec:
template:
spec:
containers:
- name: do-db-stuff
image: mysql:5
command: ["/bin/sh"]
args: ["-c", "mysql -h db -uroot -pmypassword mydb -Bse 'show tables;'"]
restartPolicy: Never
backoffLimit: 10
This job spins on a crash loop, with each pod reporting: ERROR 2005 (HY000): Unknown MySQL server host 'db' (11). If I change the command to ["/bin/sh", "-c", "sleep 3600"] and log in, I can connect from the Job's pod to the database with the command mysql -h db -uroot -pmypassword mydb -Bse 'show tables;' just fine. Based on this, I changed the command in the job to
args: ["-c", "sleep 3 && mysql -h db -uroot -pmypassword mydb -Bse 'show tables;'"]`
and now it works (i.e. correctly prints out the db tables instead of the unknown host error above). My question is why does the sleep 3 do anything? The service and database are not changing in the background at all, so why does waiting in the Job's pod make it connect?
I have GitHub repo and I want to use GithubActions to automatically execute unit Tests with every pull request.
I already set up a workflow file:
name: CI
on:
pull_request:
branches: [ master ]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
- name: Install
run: npm ci
- name: Linter
run: npm run lint
- name: Build
run: npm run build
- name: Docker
run: docker-compose up -d
- name: Wait / Sleep
uses: jakejarvis/wait-action#v0.1.0
with:
time: '10s'
- run: |
docker ps
cat ./dumps/backup.sql | docker exec -i mysql-development_1 /usr/bin/mysql -u root --password=password
- name: Test
run: npm test
As I need to insert the tables first, I want to insert a dump which does work on my machine with the exact same command used here.
However, the Action fails with this error:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
read unix #->/var/run/docker.sock: read: connection reset by peer
cat: write error: Broken pipe
##[error]Process completed with exit code 1.
How can I access the database within GithubActions?
docker-compose.yml:
version: '3'
services:
mysql-development:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: test_db
ports:
- "3308:3306"
To anyone who might run into the same problem:
https://github.blog/changelog/2020-02-21-github-actions-breaking-change-ubuntu-virtual-environments-will-no-longer-start-the-mysql-service-automatically/
You just need to start the mysql-service manually and perhaps wait for a couple of seconds
I get an "Access denied" error trying to connect to the MySQL Docker container that I start through my ansible script. If I create the container with the Docker CLI tool, everything works. I'm wondering if this has something to do with the environment variables.
This works
$ docker run --name database -e MYSQL_ROOT_PASSWORD=hunter2 -d mysql:5.7
$ mysql -h $CONTAINER_IP_ADDRESS -u root -phunter2
$ mysql>
This is broken
playbook.yml
- name: Start new MySQL container
docker:
name: database
image: mysql:5.7
state: running
env:
MYSQL_ROOT_PASSWORD=hunter2
Then the connection fails:
$ mysql -h $CONTAINER_IP_ADDRESS -u root -phunter2
ERROR 1045 (28000): Access denied for user 'root'#'$CONTAINER_IP_ADDRESS' (using password: YES)
Any idea what I'm missing here?
The env definition for the docker module should be a dict object, not a string
env:
MYSQL_ROOT_PASSWORD: hunter2
After some time I found that ansible does not play well with Docker. I've been moving some infraestructure from ansible to Docker, and keeping some legacy code like adding users to mysql or dumping dbs didn't work from Ansible.
The solution is to use shell instead:
- name: Drop/delete mysql table {{ mysql_table }}
shell: mysqladmin -h 127.0.0.1 -p{{ mysql_password }} -u{{ mysql_user }} drop {{ mysql_table }} -f
# If it fails it means db was already deleted, so continue.
ignore_errors: True
tags:
- mysql
- mysql_rebuild_db
- mysql_rebuild_db_quick
- name: Add mysql table {{ mysql_table }}
shell: mysqladmin -h 127.0.0.1 -p{{ mysql_password }} -u{{ mysql_user }} create {{ mysql_table }} -f
tags:
- mysql
- mysql_rebuild_db
- mysql_rebuild_db_quick
- name: Create mysql user {{ mysql_user }} in {{ mysql_table }}
shell: echo "CREATE USER {{ mysql_user }}#'localhost' IDENTIFIED BY '{{ mysql_password }}';" | mysql -h 127.0.0.1 -proot -uroot
# The user may already exists in a previous run
ignore_errors: True
tags:
- mysql
- mysql_rebuild_db
- mysql_rebuild_db_quick