This is my first real foray into Docker, and I'm a bit stuck at getting MySQL to play nicely with my app and I'm out of ideas on the why. I'm running the ruby:2.7.6 docker image which uses Debian. I'm trying to setup my database but I'm running into a MySQL issue.
➜ cx-core git:(master) ✗ docker compose run api bundle exec rake db:schema:load
[+] Running 2/0
⠿ Container cx-core-db-1 Running 0.0s
⠿ Container cx-core-redis-1 Running 0.0s
Starting backup.
Starting backup.
rake aborted!
failed to execute: `mysql`
Please check the output above for any errors and make sure that `mysql` is installed in your PATH and has proper permissions.
When running this command below, I am able to access the db with the credentials in my env vars. So, I'm guessing the mistake is somewhere in my docker-compose?
➜ cx-core git:(master) ✗ docker exec -it b773c7b1f960 mysql -u cx_core -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 10
Server version: 8.0.28 MySQL Community Server - GPL
Copyright (c) 2000, 2022, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
database.yml
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: <%= ENV['DB_USER'] || 'cx_core' %>
password: <%= ENV['DB_PASS'] || 'cx_core' %>
host: <%= ENV['DB_HOST'] || 'db' %>
port: 3306
development:
<<: *default
database: cx_core_dev
test:
<<: *default
database: cx_core_test
production:
<<: *default
database: cx_core_prod
dockerfile
FROM ruby:2.7.6
RUN apt-get update -qq && apt-get install -y build-essential mariadb-server mariadb-client gnupg
RUN gem install bundler
ENV GEM_HOME="/usr/local/bundle"
ENV PATH $GEM_HOME/bin:$GEM_HOME/gems/bin:$PATH
ENV APP_HOME /app
RUN mkdir $APP_HOME
WORKDIR $APP_HOME
COPY Gemfile* ./
COPY ["config.ru", "Gemfile", "Gemfile.lock", "Rakefile", "./"]
COPY /app /app
COPY config/ config/
COPY db/ db/
COPY lib/ lib/
COPY rakelib/ rakelib/
COPY bin/ bin/
EXPOSE 3000
docker-compose
version: "3.9"
services:
api:
build: .
depends_on:
- db
- redis
volumes:
- .:/app
- bundle_path:/usr/local/bundle/ # New
ports:
- "3000:3000"
environment:
- DB_HOST=db
- DB_USER=cx_core
- DB_NAME=cx_core
- DB_PASSWORD=cx_core
expose:
- 3000
command: bundle exec puma -p 3000
# entrypoint: ["./bin/start_dev.sh"]
networks:
- sinatra
db:
image: mysql
restart: always
environment:
- MYSQL_DATABASE=cx_core_dev
- MYSQL_USER=cx_core
- MYSQL_PASSWORD=cx_core
- MYSQL_ROOT_PASSWORD=cx_core
ports:
- '3306:3306'
expose:
- '3306'
volumes:
- db_data:/var/lib/mysql
networks:
- sinatra
redis:
image: 'redis:4.0-alpine'
command: redis-server --requirepass yourpassword
volumes:
- 'redis:/data'
volumes:
redis:
bundle_path: # New
db_data:
networks:
sinatra:
driver: bridge
I think you're missing step with bundle install in your Dockerfile.
Try to add this:
COPY Gemfile* ./
RUN mkdir -p ~/.ssh && ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh bundle install --jobs 2
before the line COPY ["config.ru", "Gemfile", "Gemfile.lock", "Rakefile", "./"] in your Dockerfile.
Related
it's my first time using docker so sorry if I'm making dumb mistakes. I'm trying to setup a docker container for laravel development. These are my files:
Dockerfile:
FROM php:8-fpm
COPY . /app
WORKDIR /app
COPY composer.lock composer.json ./
RUN apt-get update -y && apt-get install -y sendmail libpng-dev
RUN apt-get update && apt-get install -y \
build-essential \
curl
RUN curl -sSL https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions -o - | sh -s \
gd \
zip
RUN docker-php-ext-install mysqli pdo pdo_mysql
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
RUN composer update
RUN composer install
docker-compose.yaml:
version: '3.9'
name: caas-portal
services:
front:
build:
context: ./webapp
target: builder
ports:
- 4200:4200
volumes:
- ./webapp:/project
- /project/node_modules
mysql:
container_name: mysql
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=db
ports:
- 3307:3306/tcp
networks:
- laravel
elasticsearch:
container_name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:7.17.7
environment:
- discovery.type=single-node
ports:
- 9200:9200/tcp
- 9300:9300/tcp
volumes:
- 'elasticsearch:/usr/share/elasticsearch/data'
networks:
- laravel
app:
container_name: back
build: .
ports:
- 8000:80
networks:
- laravel
networks:
laravel:
driver: bridge
volumes:
elasticsearch:
driver: local
And this is the relevant part of my .env:
DB_CONNECTION=mysql
DB_HOST=mysql
DB_PORT=3307
DB_DATABASE=db
DB_USERNAME=root
DB_PASSWORD=password
When I run docker exec -it <container> sh and inside shell I run php artisan migrate I get the following error:
SQLSTATE[HY000] [2002] Connection refused (SQL: select * from information_schema.tables where table_schema = db and table_name = migrations and table_type = 'BASE TABLE')
I've seen similar posts, and tried to follow the answers given but still no luck. Where is the mistake, and how can it be corrected?
In your .env, try DB_PORT=3306.
According to Docker on Networking in Compose,
It is important to note the distinction between HOST_PORT and CONTAINER_PORT. [...] Networked service-to-service communication uses the CONTAINER_PORT.
If we apply this concept to your docker-compose.yaml, we know that the mysql service has a host port of 3307 while having a container port 3306.
Now if you want a your app service to connect to your mysql service (service-to-service communication), then within your app container you should connect using the mysql container port 3306, not 3307.
I have a problem with mysql 5.7 container denying access to my nodeJS container. I'm using docker-compose and I'm running docker on Ubuntu 18.04 lts.
I apologize for the mistakes in English. This is not my mother language.
I replaced all confidential information with *** and in the images I put a red line
Here's my docker-compose.yml
version: '3.1'
services:
db:
container_name: '***-db'
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
- MYSQL_ROOT_PASSWORD=***
- MYSQL_DATABASE=***
- MYSQL_USER=***
- MYSQL_PASSWORD=***
- TZ=America/Sao_Paulo
command: mysqld --sql_mode="" --character-set-server=utf8 --collation-server=utf8_slovenian_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
ports:
- 3306:3306
volumes:
- ./db_data:/var/lib/mysql
network_mode: "host"
server:
image: server
restart: always
ports:
- "2000:22"
- "3000:3000"
network_mode: "host"
front:
image: portal
restart: always
links:
- server:server
ports:
- 4200:80
I am using a volume with the database for my mysql container. In this volume I have my user and password set and I also have my schema with every table. I copy everything from this path:
/var/lib/mysql
that I have on my configuration (I am not using docker on this one) in another computer with the same version of mysql and that I know it is working.
Here is my server Dockerfile:
FROM ubuntu:18.04
LABEL maintainer = "***"
LABEL build_date="2020-04-24"
RUN apt-get update && apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get update && apt-get install -y nodejs
#COPY AN CONFIGURATION OF APP
COPY ***-server /home/root/***-server
WORKDIR /home/root/***-server
RUN npm install
EXPOSE 22
EXPOSE 3000
CMD ["npm", "start"]
Here is my nodeJS file to connect with the mysql:
"use strict";
module.exports = {
config: {
connectionLimit : 200,
host: 'localhost',
user : '***',
password : '***',
database: '***'
}
}
this is the error mesage that docker compose send to me, when I try to connect:
And this is image of the container of the mysql that I am trying to access. I make access to the mysql inside the container using the same user that I am using on the server.
I'm trying to have a docker container use my local mysql database. Using docker-compose up, I am able to get all the pieces running, however, the db exits with status code 1, and the rails application cannot connect to any database instance. I know that mysql is running on my system and that the service has been started. My docker-compose file:
version: '3'
services:
db:
image: mysql
restart: always
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: db
MYSQL_USER: root
MYSQL_PASSWORD: password
volumes:
- ./tmp/db:/var/lib/mysql
web:
build: .
image: rails-devise:1.0.0
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- ".:/baas-status"
ports:
- "3000:3000"
depends_on:
- db
I know the containers are running because docker ps gives the following output:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a49aba0cd68 rails-devise:1.0.0 "entrypoint.sh bash …" 23 minutes ago Up 2 minutes 0.0.0.0:3000->3000/tcp status_web_1
38554baf5efc mysql "docker-entrypoint.s…" 23 minutes ago Up 2 minutes 3306/tcp, 0.0.0.0:3002->3002/tcp, 33060/tcp status_db_1
Even though I exposed port 3002 and want to use that as the port for the database, is it unable to connect because I'm on the wrong port? I see in the docker ps output that mysql is on port 3306.
My database.yml file:
base: &base
adapter: mysql2
encoding: utf8
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
default: &default
<<: *base
username: <%= ENV['DB_USERNAME'] %>
password: <%= ENV['DB_PASSWORD'] %>
host: <%= ENV['DB_HOST'] %>
database: <%= ENV['DB_NAME'] %>
development:
<<: *base
username: root
password: password
host: db
database: db
I have also tried editing the host for development to be host: db. That didn't change anything either. I tried running docker-compose run web rake db:create as well, and I get the following error:
This has been fixed with the current changes to the docker-compose file
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
At this point, I'm at a loss on where to proceed. Any information or help would be greatly appreciated. Thank you.
Edit:
I have changed the ports for the db and it seems to have fixed the connection issues. Now I'm running into an error that gives
web_1 | Mysql2::Error::ConnectionError (Plugin caching_sha2_password could not be loaded: /usr/lib/x86_64-linux-gnu/mariadb19/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory):
I'm going to do some research to figure out the meaning behind this error, any tips to move in the right direction would be greatly appreciated though!
Edit2:
I ended up connecting the db by reverting mysql back to 5.7.18 to avoid the sha authentication. The docker-compose file and settings that worked for me are as follows:
docker-compose.yml
version: '2'
services:
db:
image: mysql:5.7.18
restart: always
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
volumes:
- db-data:/var/lib/mysql
web:
build: .
image: rails-devise:1.0.0
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- ".:/baas-status"
ports:
- 3000:3000
links:
- db
depends_on:
- db
volumes:
db-data:
driver: local
The database.yml file remained the same with the exception of changing the password to root. This is the solution that worked for me.
Resource I ended up using!
Sounds like you figured out the connection issue. You'll want to connect on 3306, not 3002 based on the docker-compose file you posted.
I assume you also made sure to define these environment variables in your rails docker container as well. Is that correct?
username: <%= ENV['DB_USERNAME'] %>
password: <%= ENV['DB_PASSWORD'] %>
host: <%= ENV['DB_HOST'] %>
database: <%= ENV['DB_NAME'] %>
Since you do not define a tag of the mysql image to use in your docker-compose file, it will pull the latest image which is mysql ~8.x. MySQL 8 uses the caching_sha2_password authentication plugin by default (unless you specify otherwise). The Rails mysql2 driver version you're using likely does not support this authentication mechanism yet.
So, if you would like to revert to an older authentication plugin (like the one from mysql 5.7), you can use a command line flag in your mysql container like so:
services:
...
db:
image: mysql:8.0
...
command: mysqld --default-authentication-plugin=mysql_native_password
More information can also be found on this SO answer.
environment
- MacOS Mojave 10.14.6
- Ruby 2.6.4
- Rails 5.2.2
- mysql Ver 8.0.17 for osx10.14 on x86_64 (Homebrew)
- Docker 2.1.0.3
What I want to realize
I want to clear the following error.
Mysql2::Error::ConnectionError (Unknown MySQL server host 'db' (0)):
Terminal
$ rails s
=> Booting Puma
=> Rails 5.2.2 application starting in development
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.12.1 (ruby 2.6.4-p104), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://localhost:3000
Use Ctrl-C to stop
Started GET "/" for ::1 at 2019-10-07 22:11:28 +0900
Mysql2::Error::ConnectionError (Unknown MySQL server host 'db' (0)):
...
$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------
rails_test_db_1 docker-entrypoint.sh mysqld Up 3306/tcp, 33060/tcp
rails_test_web_1 bundle exec rails s -p 300 ... Exit 1
docker-compose.yml
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- 3000:3000
depends_on:
- db
tty: true
stdin_open: true
db:
image: mysql:5.7
volumes:
- db-volume:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: password
volumes:
db-volume:
database.yml
# MySQL. Versions 5.0 and up are supported.
#
# Install the MySQL driver
# gem install mysql2
#
# Ensure the MySQL gem is defined in your Gemfile
# gem 'mysql2'
#
# And be sure to use new-style password hashing:
# http://dev.mysql.com/doc/refman/5.7/en/old-client.html
#
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: password
host: db
development:
<<: *default
database: app_development
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: app_test
# As with config/secrets.yml, you never want to store sensitive information,
# like your database password, in your source code. If your source code is
# ever seen by anyone, they now have access to your database.
#
# Instead, provide the password as a unix environment variable when you boot
# the app. Read http://guides.rubyonrails.org/configuring.html#configuring-a-database
# for a full rundown on how to provide these environment variables in a
# production deployment.
#
# On Heroku and other platform providers, you may have a full connection URL
# available as an environment variable. For example:
#
# DATABASE_URL="mysql2://myuser:mypass#localhost/somedatabase"
#
# You can use this database configuration with:
#
# production:
# url: <%= ENV['DATABASE_URL'] %>
#
production:
<<: *default
database: app_production
username: app
password: <%= ENV['APP_DATABASE_PASSWORD'] %>
That error is telling you that the host db is unknown.
You are running rails s on your terminal. That is starting your rails server outside of docker, therefore the host db cannot be resolved. (You only have that kind of DNS resolution for the containers running within the same docker network)
You should just run docker-compose up so that both services defined in your docker-compose.yml (web and db) start in the same network (docker-compose takes care of that, you don't need to do any extra setup for it)
You can create a network that will be shared inside your docker between images
networks:
my_network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.25.0.1/16
gateway: 172.25.0.1
Then assign specific IP to your db container:
networks:
my_network:
ipv4_address: 172.25.0.3
After that you need to make sure db hostname is linked with db container IP.
The code bellow will add new line 172.25.0.3 db to you /etc/hosts file.
extra_hosts:
- 'db:172.25.0.3'
Final version of docker-compose will look like this:
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- 3000:3000
depends_on:
- db
tty: true
stdin_open: true
extra_hosts:
- 'db:172.25.0.3'
db:
image: mysql:5.7
volumes:
- db-volume:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: password
networks:
my_network:
ipv4_address: 172.25.0.3
volumes:
db-volume:
networks:
my_network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.25.0.1/16
gateway: 172.25.0.1
If you do this you can be sure that db host will be always available. I hope that will help you.
I'm newbie using docker and my issue is to load a Rails 4 application using MySQL and Docker with environment variables.
Basically, I build the app and run it. In another terminal I create mysql database and user/password to be used by the database.yml. Here are the steps that I run:
$ docker-compose build
$ docker-compose up -d
$ docker exec -it poseidon_db_1 bash -l
#root#dockerimg mysql -uroot -proot
mysql> create database poseidon_development;
mysql> CREATE USER 'poseidon_user'#'localhost' IDENTIFIED BY 'poseidon_password';
mysql> CREATE USER 'poseidon_user'#'%' IDENTIFIED BY 'poseidon_password';
mysql> GRANT ALL ON poseidon_development.* TO 'poseidon_user'#'localhost';
mysql> GRANT ALL ON poseidon_development.* TO 'poseidon_user'#'%';
mysql> FLUSH PRIVILEGES;
mysql> exit
#root#dockerimg exit
$ docker-compose run web bundle exec rake db:migrate
database.yml
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: <%= ENV["DATABASE_USERNAME"] %>
password: <%= ENV["DATABASE_PASSWORD"] %>
database: <%= ENV["DATABASE_NAME"] %>
host: db
development:
<<: *default
test:
<<: *default
docker-compose.yml
version: '3.6'
services:
db:
restart: always
image: mysql:5.6.34
ports:
- "3306:3306"
volumes:
- dbdata
env_file:
- .env
dbdata:
image: tianon/true
volumes:
- /var/lib/mysql
web:
build: .
ports:
- '3000:3000'
volumes:
- webdata
links:
- "db"
env_file:
- .env
webdata:
image: tianon/true
volumes:
- ".:/var/www/poseidon"
Dockerfile
FROM ruby:2.3.7
ENV LANG C.UTF-8
RUN apt-get update -qq
RUN apt-get install -y build-essential mysql-client vim
WORKDIR /tmp
ADD Gemfile Gemfile
ADD Gemfile.lock Gemfile.lock
RUN bundle install
ENV APP_ROOT /var/www/poseidon
RUN mkdir -p $APP_ROOT
WORKDIR $APP_ROOT
COPY . $APP_ROOT
RUN chmod a+x .env
EXPOSE 3000
ENTRYPOINT ["bundle", "exec"]
CMD ["rails", "server", "-p", "3000", "-b", "0.0.0.0"]
.env
SECRET_KEY_BASE=xxxxxxxx
RAILS_ENV=development
RAILS_SERVE_STATIC_FILES=true
URL_OPTIONS=domain.com
RECAPTCHA_SECRET_KEY=xxxxxxx
RECAPTCHA_SITE_KEY=xxxxxxx
MAILGUN_API=xxxxxx
MAILGUN_DOMAIN=xxxxxxx
MYSQL_ROOT_PASSWORD=root
DATABASE_USERNAME=poseidon_user
DATABASE_PASSWORD=poseidon_password
DATABASE_NAME=poseidon_development
The problem is, when accessing http://127.0.0.1, I receive Access denied for user '<%= ENV["DATABASE_USERNAME"] %>'#'IP' (using password: YES)
I don't know what to do anymore, it seems to not load the env vars correctly.
Any help will be really appreciated!
Regards
I was stumped on this issue for a day or two. Things were working fine out of Docker but for some reason within the container the ERB wasn't being parsed.
This post seemed to help: rails database.yml not accepting ERB (which I found via Rails not replace ENV's value in database.yml)
I just experienced the same thing, and came across your post. I had
been following a tutorial that had me create a puma.conf file that
contained the code below:
ActiveRecord::Base.establish_connection( YAML.load_file( "#{app_dir}/config/database.yml" )[rails_env])
I modified to the following, and everything worked as expected:
require 'erb'
ActiveRecord::Base.establish_connection( YAML.load( ERB.new( File.read( "#{app_dir}/config/database.yml" )).result)[rails_env])