When I execute the xdcomp_my_sql_client command it pings the ip but then when it tries to reach the mysql server it fails. If I do the exact same command once the container is running it works. It seems that the mysql server is not running at the moment the command is executed. But I have used the "depends_on" command, so what I'm doing wrong?
Thank you.
version: '2'
services:
xdcomp_my_sql_server:
image: mysql/mysql-server:latest
environment:
MYSQL_ROOT_PASSWORD: diego
MYSQL_USER: otro
MYSQL_PASSWORD: otro
MYSQL_ROOT_HOST: 172.28.0.101
networks:
SQLNetwork:
ipv4_address: 172.28.0.102
xdcomp_my_sql_client:
build: .
command: sh -c 'ping -c 5 172.28.0.102 && mysql -h 172.28.0.102 -u root -pdiego sys < /lafayette/forensic.sql && tail -f /etc/hostname'
ports:
- 83:80
networks:
SQLNetwork:
ipv4_address: 172.28.0.101
depends_on:
- xdcomp_my_sql_server
networks:
SQLNetwork:
driver: "bridge"
ipam:
config:
- subnet: 172.28.0.0/24
gateway: 172.28.0.201
Client Dockerfile
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install -y mysql-client
RUN apt-get install -y git
RUN git clone https://github.com/linkedin/lafayette
RUN apt-get update
RUN apt-get install -y python-dnspython
RUN apt-get install -y python-pip
RUN pip install Flask
RUN pip install python-dateutil
RUN apt-get install -y python-mysqldb
RUN pip install requests
RUN pip install multiprocessing
RUN pip install multiprocess
RUN apt-get install -y vim
RUN apt-get -y install iputils-ping
Working with docker includes 2 steps:
building a image
creating container based on image and running this container
The point is you mysql client can connect to server only on second step.
Dockerfile "executes" while you build image. In this time you don't have any containers, therefore you can`t connect to them.
You need move command
RUN cd lafayette && mysql –h 172.25.0.102 –u root –ppass sys < forensic.sql
from Dockerfile to entrypoint.sh
Can you show me your Dockerfile? I will try to fix the issue
The problem was the following: the mysql server was not running at the time the client command was executed. Even when the key "depends_on" was present (see:https://docs.docker.com/compose/startup-order/) For that reason the solution was to wait for the mysql server. This was done with the following command:
command: sh -c 'until nc -z -v -w20 172.28.0.102 3306; do sleep 1;
echo "Waiting for mysqlserver to come up..."; done && ping -c 5 172.28.0.102 &&
mysql -h 172.28.0.102 -u root -pdiego sys < /lafayette/forensic.sql &&
tail -f /etc/hostname'
The wait for was made by the following line:
until nc -z -v -w20 172.28.0.102 3306; do sleep 1;
echo "Waiting for mysqlserver to come up..."; done
nc= netcat command, it check the connection to a given ip and port
Related
i just have push my Symfony 5.4 application in my company server with Gitlab-ci and docker.
I have a Dokcerfile in the root avec my projet:
FROM php:7.4-apache
ENV APACHE_DOCUMENT_ROOT /var/www/html/public
RUN sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf
COPY . /var/www/html/
RUN apt-get update \
&& apt-get install -y --no-install-recommends locales apt-utils git libicu-dev g++ libpng-dev libxml2-dev libzip-dev libonig-dev libxslt-dev;
RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && \
echo "fr_FR.UTF-8 UTF-8" >> /etc/locale.gen && \
locale-gen
# INSTALL COMPOSER
RUN curl -sSk https://getcomposer.org/installer | php -- --disable-tls && \
mv composer.phar /usr/local/bin/composer
# INSTALL EXTENSION PHP
RUN docker-php-ext-configure intl
RUN docker-php-ext-install pdo pdo_mysql gd opcache intl zip calendar dom mbstring zip gd xsl
RUN pecl install apcu && docker-php-ext-enable apcu
# INSTALL YARN
RUN apt-get update && apt-get install -y gnupg2
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y yarn
RUN apt update
RUN chown -R www-data:www-data /var/www/html
RUN a2enmod rewrite
RUN service apache2 restart
And i have this job on my gitlab pipeline:
deploy_image:
stage: deploy
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- cd /srv/docker/stacks/r-panda
- docker network disconnect --force r-panda_default traefik_app
- /usr/local/bin/docker-compose down
- /usr/local/bin/docker-compose pull
- /usr/local/bin/docker-compose up -d
- docker network connect r-panda_default traefik_app
after_script:
- docker exec r-panda_app yarn install
- docker exec r-panda_app yarn run build
- ls -al
environment:
name: dev
url: $SERVER_URL
variables:
DATABASE_URL: $DATABASE_URL
tags:
- deploy
And there is my docker-compose.yaml on the server who run the contener with the application:
version: '3.9'
services:
r-panda_app:
container_name: r-panda_app
image: panda
environment:
- DATABASE_URL="mysql://db_user:db_pass#bbd_server:3306/panda_bdd"
restart: always
labels:
- traefik.enable=true
- traefik.http.routers.router-r-panda.rule=Host(`r-panda.fr`)
- traefik.http.routers.router-r-panda.tls.certresolver=myresolver
- traefik.http.routers.router-r-panda.tls=true
With this config i have the following error: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directory
But if i change my config/packages/doctrine.yaml
This
doctrine:
dbal:
driver: 'pdo_mysql'
server_version: '5.7'
charset: utf8mb4
url: '%env(resolve:DATABASE_URL)%'
To this:
doctrine:
dbal:
driver: 'pdo_mysql'
server_version: '5.7'
charset: utf8mb4
url: 'the real path for the DATABASE_URL'
Everything works
I don't know what to do, can someone help me
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)
Hi guys !
I have a little (big) issue with my gitlab-ci.yml :(
I'm trying to run my tests PHPUnit on the pipeline but when I want to create a mysql database and to connect it, it fails.
This is my .gitlab-ci.yml :
image: jakzal/phpqa:php8.0
before_script:
- composer install
stages:
- SecurityChecker
- UnitTests
cache:
paths:
- vendor/
- ~/.composer/cache/files
#On teste qu'il n'y a pas de faille de sécurité dans les différentes librairies
security-checker:
stage: SecurityChecker
script:
- local-php-security-checker composer.lock
allow_failure: false
#On lance les tests unitaires PHP UNIT
phpunit:
stage: UnitTests
services:
- name: mysql:latest
before_script:
- apt-get update && apt-get dist-upgrade && apt-get install -y git libzip-dev autoconf build-essential pkg-config libmariadb-dev-compat libmariadb-dev default-mysql-client mariadb-client mariadb-server
- curl -sSk https://getcomposer.org/installer | php -- --disable-tls && mv composer.phar /usr/local/bin/composer
- docker-php-ext-install mysqli pdo pdo_mysql zip
# Connect MySQL
- echo "SET PASSWORD FOR 'root'#'localhost' = PASSWORD('password');" | mysql -u root
- echo "SELECT 'OK';" | mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -h mysql "$MYSQL_DATABASE"
# Import BDD tests
- mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -h localhost < "database.sql"
script:
- vendor/bin/phpunit --testdox
allow_failure: false
This is the result :
Pipeline error
Do you have any ideas fo my issue ?
I didn't found anything on the different forums for now... :sleepy:
Thanks a lot !
Take a look here: https://docs.gitlab.com/ee/ci/services/
I was able to get it work like this:
stages:
- test
services:
- mariadb
test:
stage: test
image: mariadb
variables:
MARIADB_ROOT_PASSWORD: password
MARIADB_DATABASE: dbname
script:
- mysql --user=root --password="$MARIADB_ROOT_PASSWORD" --host=mariadb
Might you have to change -h localhost to -h mysql.
The answer to my issue is this code !
It works for me :)
#On lance les tests unitaires PHP UNIT
phpunit:
stage: UnitTests
services:
- name: mysql:8.0.11
variables:
MYSQL_DATABASE: 'database-name'
MYSQL_ROOT_PASSWORD: ''
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
DB_USERNAME: 'runner'
DB_PASSWORD: 'password'
DB_HOST: 'mysql'
before_script:
# Install packages and docker
- apt-get update -y && apt-get dist-upgrade -y && apt-get install -y git libzip-dev autoconf curl libmcrypt-dev build-essential pkg-config libmariadb-dev-compat libmariadb-dev default-mysql-client mariadb-client mariadb-server
- docker-php-ext-install mysqli pdo pdo_mysql zip
# Config MySQL
- rm -f /tmp/mysql.sock.lock
- sleep 10
# Import BDD for tests units
- mysql -u root -h "$DB_HOST" < "web/files/database-test.sql"
# Check BDD
- echo "SHOW tables;" | mysql -u root -h "$DB_HOST" -D "$MYSQL_DATABASE"
script:
- vendor/bin/phpunit --testdox
allow_failure: false
I have a complex Dockerfile which install much more than just mysql-server so I cannot start from an existing mysql container.
When removing all the extra-stuff I get this Dockerfile
FROM ubuntu:latest
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_ROOT_USER=root
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get install -y apt-utils
RUN apt-get install -y mysql-server
RUN usermod -d /var/lib/mysql/ mysql
RUN service mysql start
Unfortunately, mysql does not want to start:
---> 57a66bd64c2c
Step 8/9 : RUN usermod -d /var/lib/mysql/ mysql
---> Running in 596df248c2e4
---> ee78442bcc56
Step 9/9 : RUN service mysql start
---> Running in 0d9e5803cf33
* Starting MySQL database server mysqld
...fail!
The command '/bin/sh -c service mysql start' returned a non-zero code: 1
What is my mistake?
Looks like you've removed the most important parts of your docker file. Here is the Official MySQL repo Docker file.
FROM oraclelinux:7-slim
ENV PACKAGE_URL https://repo.mysql.com/yum/mysql-8.0-community/docker/x86_64/mysql-community-server-minimal-8.0.2-0.1.dmr.el7.x86_64.rpm
# Install server
RUN rpmkeys --import http://repo.mysql.com/RPM-GPG-KEY-mysql \
&& yum install -y $PACKAGE_URL \
&& yum install -y libpwquality \
&& rm -rf /var/cache/yum/*
RUN mkdir /docker-entrypoint-initdb.d
VOLUME /var/lib/mysql
COPY docker-entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
EXPOSE 3306 33060
CMD ["mysqld"]
You need to include a proper source with correct version to pull the image from.
and expose right ports, separate out volumes for MySQL to run. your container maybe failing due to any of this. I'd say remove the MySQL part out of your dockerfile and run the rest of the container.
Use the official mySQL image and install it in separate container. and then you can connect the Database with other apps.
I am creating developer environment for rails app using docker, docker-compose and docker-sync. When I run docker-compose up first time it builds the image of my rails app and downloads mysql image, but not able to connect it, but if I quit this and run docker-compose up again everything works fine. Here are my docker configurations
Dockerfile:
FROM ruby:2.5.1
RUN apt-get update && \
apt-get -y install sudo
RUN apt-get update && apt-get install -y \
build-essential \
nodejs \
vim \
mysql-client \
apt-transport-https \
build-essential
RUN curl -s -N https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -\
&& echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list \
&& apt-get update \
&& apt-get install -y yarn
RUN mkdir /rails-app
ENV RAILS_HOME /rails-app
WORKDIR /tmp
ADD Gemfile Gemfile
ADD Gemfile.lock Gemfile.lock
RUN bundle install
WORKDIR /rails-app
ADD . /rails-app
EXPOSE 3000
CMD ["bash"]
Below is my docker-compose
docker-compose.yml
version: '3'
services:
web:
image: "rails_app"
build: .
ports:
- "3000:3000"
env_file:
- web_env
- db_env
depends_on:
- db
command: >
bash -c "bundle exec rails db:schema:load &&
bundle exec rails db:seed &&
rm -f tmp/pids/server.pid &&
bundle exec rails s -p 3000 -b '0.0.0.0'"
db:
image: library/mysql:5.6.42
env_file:
- db_env
docker-compose dev file for volume syncing
docker-compose-dev.yml
version: '3'
services:
web:
volumes:
- rails_app_volume_sync:/rails-app:nocopy
volumes:
rails_app_volume_sync:
external: true
docker-sync file
docker-sync.yml
version: "2"
syncs:
rails_app_volume_sync:
src: '.'
sync_host_ip: 'auto'
sync_strategy: 'unison'
notify_terminal: true
environment files:
web_env:
DATABASE_HOST=db
RAILS_ENV=development
db_env:
MYSQL_DATABASE=development_exp
MYSQL_ROOT_PASSWORD=root
MYSQL_USER=dev
MYSQL_PASSWORD=dev
and my database config is as follows:
development:
adapter: mysql2
encoding: utf8
database: <%= ENV['MYSQL_DATABASE'] %>
username: <%= ENV['MYSQL_USER'] %>
password: <%= ENV['MYSQL_PASSWORD'] %>
host: <%= ENV['DATABASE_HOST'] %>
reconnect: true
pool: 5
when I run docker-sync-stack start I get the following error after the images are build.
web_1 | Mysql2::Error::ConnectionError: Can't connect to MySQL server on 'db' (111 "Connection refused")
if I stop the containers and run it again, its working, why is it failing first time ?
The problem is that you aren't waiting for the database to be ready before running the migrations. depends_on only guarantee that your mysql process in the container is started but it has no way to know when the mysql process has initialized and available to process new connections.
A solution can be to wait for the TCP port to be up, some use wait-for-it for it. Another solution is to simply fail and restart the container until the database is ready (set the Restart Policy of the container to always).
The go app simply inserts a harcoded value in a mysql table and spits it back out. This is done using this database driver. It works fine on linux servers, but during gitlab's CI this returns:
dial tcp 127.0.0.1:3306: connect: connection refused
This is the .gitlab-ci.yml
image: mysql
services:
- mysql:latest
variables:
MYSQL_DATABASE: storage
MYSQL_ROOT_PASSWORD: root
MYSQL_HOST: mysql
MYSQL_USER: root
MYSQL_TRANSAPORT: tcp
MYSQL_ADDRESS: "127.0.0.1:3306"
job:
script:
- apt-get update -qq && apt-get install -qq curl && apt-get install -qq git
- echo "SHOW GLOBAL VARIABLES LIKE 'PORT';" | mysql --user="$MYSQL_USER" --password="$MYSQL_ROOT_PASSWORD" --host="$MYSQL_HOST" "$MYSQL_DATABASE"
- curl -O https://dl.google.com/go/go1.10.1.linux-amd64.tar.gz
- tar -C /usr/local -xzf go1.10.1.linux-amd64.tar.gz
- rm go1.10.1.linux-amd64.tar.gz
- echo "export PATH=\$PATH:/usr/local/go/bin" >> ~/.bashrc
- echo "export GOPATH=\$HOME/go" >> ~/.bashrc
- echo "export PATH=\$PATH:\$GOPATH/bin" >> ~/.bashrc
- source ~/.bashrc
- go get github.com/go-sql-driver/mysql
- go build main.go
- ./main
Is there a standard way to use mysql from golang during CI?