I'm trying to have a docker container use my local mysql database. Using docker-compose up, I am able to get all the pieces running, however, the db exits with status code 1, and the rails application cannot connect to any database instance. I know that mysql is running on my system and that the service has been started. My docker-compose file:
version: '3'
services:
db:
image: mysql
restart: always
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: db
MYSQL_USER: root
MYSQL_PASSWORD: password
volumes:
- ./tmp/db:/var/lib/mysql
web:
build: .
image: rails-devise:1.0.0
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- ".:/baas-status"
ports:
- "3000:3000"
depends_on:
- db
I know the containers are running because docker ps gives the following output:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a49aba0cd68 rails-devise:1.0.0 "entrypoint.sh bash …" 23 minutes ago Up 2 minutes 0.0.0.0:3000->3000/tcp status_web_1
38554baf5efc mysql "docker-entrypoint.s…" 23 minutes ago Up 2 minutes 3306/tcp, 0.0.0.0:3002->3002/tcp, 33060/tcp status_db_1
Even though I exposed port 3002 and want to use that as the port for the database, is it unable to connect because I'm on the wrong port? I see in the docker ps output that mysql is on port 3306.
My database.yml file:
base: &base
adapter: mysql2
encoding: utf8
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
default: &default
<<: *base
username: <%= ENV['DB_USERNAME'] %>
password: <%= ENV['DB_PASSWORD'] %>
host: <%= ENV['DB_HOST'] %>
database: <%= ENV['DB_NAME'] %>
development:
<<: *base
username: root
password: password
host: db
database: db
I have also tried editing the host for development to be host: db. That didn't change anything either. I tried running docker-compose run web rake db:create as well, and I get the following error:
This has been fixed with the current changes to the docker-compose file
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
At this point, I'm at a loss on where to proceed. Any information or help would be greatly appreciated. Thank you.
Edit:
I have changed the ports for the db and it seems to have fixed the connection issues. Now I'm running into an error that gives
web_1 | Mysql2::Error::ConnectionError (Plugin caching_sha2_password could not be loaded: /usr/lib/x86_64-linux-gnu/mariadb19/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory):
I'm going to do some research to figure out the meaning behind this error, any tips to move in the right direction would be greatly appreciated though!
Edit2:
I ended up connecting the db by reverting mysql back to 5.7.18 to avoid the sha authentication. The docker-compose file and settings that worked for me are as follows:
docker-compose.yml
version: '2'
services:
db:
image: mysql:5.7.18
restart: always
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
volumes:
- db-data:/var/lib/mysql
web:
build: .
image: rails-devise:1.0.0
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- ".:/baas-status"
ports:
- 3000:3000
links:
- db
depends_on:
- db
volumes:
db-data:
driver: local
The database.yml file remained the same with the exception of changing the password to root. This is the solution that worked for me.
Resource I ended up using!
Sounds like you figured out the connection issue. You'll want to connect on 3306, not 3002 based on the docker-compose file you posted.
I assume you also made sure to define these environment variables in your rails docker container as well. Is that correct?
username: <%= ENV['DB_USERNAME'] %>
password: <%= ENV['DB_PASSWORD'] %>
host: <%= ENV['DB_HOST'] %>
database: <%= ENV['DB_NAME'] %>
Since you do not define a tag of the mysql image to use in your docker-compose file, it will pull the latest image which is mysql ~8.x. MySQL 8 uses the caching_sha2_password authentication plugin by default (unless you specify otherwise). The Rails mysql2 driver version you're using likely does not support this authentication mechanism yet.
So, if you would like to revert to an older authentication plugin (like the one from mysql 5.7), you can use a command line flag in your mysql container like so:
services:
...
db:
image: mysql:8.0
...
command: mysqld --default-authentication-plugin=mysql_native_password
More information can also be found on this SO answer.
Related
I’m on Mac OS Big Sur and running the following Docker versions
$ docker -v
Docker version 20.10.12, build e91ed57
$ docker-compose -v
Docker Compose version v2.2.3
I’m running a Rails 6 app, whose database is configured like so
development:
adapter: mysql2
encoding: utf8
host: host.docker.internal
database: cfs
pool: 5
username: myuser
password: bypass
I have to use “host.docker.internal” so that the Rails app can access the Docker db when running inside of Docker, set up in my docker-compose.yml like so
services:
db:
image: mysql:5.7
command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
restart: always
volumes:
- ./docker/provision/mysql/init:/docker-entrypoint-initdb.d
environment:
MYSQL_ROOT_PASSWORD: bypass
ports:
# <Port exposed> : < MySQL Port running inside container>
- '3306:3306'
expose:
# Opens port 3306 on the container
- '3306'
…
web:
#restart: always
build: ./my-project
ports:
- "3000:3000"
expose:
- '3000'
command: foreman start
volumes:
- ./my-project/:/app
depends_on:
- db
However if I run the app locally without Docker, I have to change my config files to remove “host.docker.internal” and use “127.0.0.1” instead (or localhost).
Is there a way I can set things up so that I have a single database config file that works in both Docker and without Docker such that I don’t have to change the host around?
You can use ERB markup in the database.yml file, which lets you use an environment variable here. In general I'd suggest making default values for things be whatever default will work in a plain non-Docker development environment.
# config/database.yml
development:
host: <%= ENV['DATABASE_HOST'] || 'localhost' %>
If you're running this in a setup where the database hostname isn't localhost – it's in a sibling container, or you're using a cloud-hosted database like Amazon RDS – you can set that environment variable.
# docker-compose.yml
version: '3.8'
services:
db:
image: 'mysql:5.7'
et: cetera
web:
build: .
ports:
- '3000:3000'
depends_on:
- db
environment:
DATABASE_HOST: db # <-- will be read in database.yml
I am running into this error with my Rails + MySQL Docker setup:
Mysql2::Error::ConnectionError: Unknown MySQL server host 'db' (-2)
/usr/local/bundle/gems/mysql2-0.5.2/lib/mysql2/client.rb:90:in `connect'
/usr/local/bundle/gems/mysql2-0.5.2/lib/mysql2/client.rb:90:in `initialize'
my docker-compose.yml:
version: '3.7'
services:
db:
# https://github.com/passbolt/passbolt_docker/issues/103
image: mysql:5.7
restart: always
healthcheck:
test: ["CMD-SHELL", 'mysql --database=$$MYSQL_DATABASE --password=$$MYSQL_ROOT_PASSWORD --execute="SELECT count(table_name) > 0 FROM information_schema.tables;" --skip-column-names -B']
interval: 30s
timeout: 10s
retries: 4
environment:
MYSQL_DATABASE: 'db'
MYSQL_USER: 'user'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3305:3306"
expose:
- '3305'
volumes:
- my-db:/var/lib/mysql
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- '.:/app'
ports:
- '3000:3000'
environment:
DB_PORT: 3306
DB_HOST: db
DATABASE_URL: mysql2://user:password#db:3306
depends_on:
- db
volumes:
my-db:
and my database.yml:
default: &default
adapter: mysql2
encoding: utf8mb4
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
username: user
password: password
host: localhost
url: <%= ENV['DATABASE_URL'] %>
port: 3305
According to this, I'm supposed to wait for MySQL to start? However I added a health check in the healthcheck section of the docker-compose file and it didn't work.
I also tried to use netcat to check the port but it also didn't work. How come I can't connect to the host db from my Docker web container? What am I doing wrong?
Turns out for my problem, I needed to clear out my docker volumes and recreate everything. MySQL was experiencing an error while booting up.
Basically ran this:
docker-compose down
docker system prune --force --volumes
And then restarted everything, ensuring that MySQL ("db") was running successfully before trying to connect to db.
This message is complaining about a missing "MYSQL host" called 'db'
Mysql2::Error::ConnectionError: Unknown MySQL server host 'db' (-2)
According to your docker-compose file
environment:
MYSQL_DATABASE: 'db'
MYSQL_USER: 'user'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
You should choose here a valid parameters (use ENV variables even better).
Also make sure
you can access MYSQL from console with the specified credintials.
The specified MYSQL user has access from external host ( docker is will not be on the same network )
environment
- MacOS Mojave 10.14.6
- Ruby 2.6.4
- Rails 5.2.2
- mysql Ver 8.0.17 for osx10.14 on x86_64 (Homebrew)
- Docker 2.1.0.3
What I want to realize
I want to clear the following error.
Mysql2::Error::ConnectionError (Unknown MySQL server host 'db' (0)):
Terminal
$ rails s
=> Booting Puma
=> Rails 5.2.2 application starting in development
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.12.1 (ruby 2.6.4-p104), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://localhost:3000
Use Ctrl-C to stop
Started GET "/" for ::1 at 2019-10-07 22:11:28 +0900
Mysql2::Error::ConnectionError (Unknown MySQL server host 'db' (0)):
...
$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------
rails_test_db_1 docker-entrypoint.sh mysqld Up 3306/tcp, 33060/tcp
rails_test_web_1 bundle exec rails s -p 300 ... Exit 1
docker-compose.yml
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- 3000:3000
depends_on:
- db
tty: true
stdin_open: true
db:
image: mysql:5.7
volumes:
- db-volume:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: password
volumes:
db-volume:
database.yml
# MySQL. Versions 5.0 and up are supported.
#
# Install the MySQL driver
# gem install mysql2
#
# Ensure the MySQL gem is defined in your Gemfile
# gem 'mysql2'
#
# And be sure to use new-style password hashing:
# http://dev.mysql.com/doc/refman/5.7/en/old-client.html
#
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: password
host: db
development:
<<: *default
database: app_development
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: app_test
# As with config/secrets.yml, you never want to store sensitive information,
# like your database password, in your source code. If your source code is
# ever seen by anyone, they now have access to your database.
#
# Instead, provide the password as a unix environment variable when you boot
# the app. Read http://guides.rubyonrails.org/configuring.html#configuring-a-database
# for a full rundown on how to provide these environment variables in a
# production deployment.
#
# On Heroku and other platform providers, you may have a full connection URL
# available as an environment variable. For example:
#
# DATABASE_URL="mysql2://myuser:mypass#localhost/somedatabase"
#
# You can use this database configuration with:
#
# production:
# url: <%= ENV['DATABASE_URL'] %>
#
production:
<<: *default
database: app_production
username: app
password: <%= ENV['APP_DATABASE_PASSWORD'] %>
That error is telling you that the host db is unknown.
You are running rails s on your terminal. That is starting your rails server outside of docker, therefore the host db cannot be resolved. (You only have that kind of DNS resolution for the containers running within the same docker network)
You should just run docker-compose up so that both services defined in your docker-compose.yml (web and db) start in the same network (docker-compose takes care of that, you don't need to do any extra setup for it)
You can create a network that will be shared inside your docker between images
networks:
my_network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.25.0.1/16
gateway: 172.25.0.1
Then assign specific IP to your db container:
networks:
my_network:
ipv4_address: 172.25.0.3
After that you need to make sure db hostname is linked with db container IP.
The code bellow will add new line 172.25.0.3 db to you /etc/hosts file.
extra_hosts:
- 'db:172.25.0.3'
Final version of docker-compose will look like this:
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- 3000:3000
depends_on:
- db
tty: true
stdin_open: true
extra_hosts:
- 'db:172.25.0.3'
db:
image: mysql:5.7
volumes:
- db-volume:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: password
networks:
my_network:
ipv4_address: 172.25.0.3
volumes:
db-volume:
networks:
my_network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.25.0.1/16
gateway: 172.25.0.1
If you do this you can be sure that db host will be always available. I hope that will help you.
The full error is Doctrine\DBAL\Exception\ConnectionException: An exception occurred in driver: SQLSTATE[HY000] [2002] No such file or directory in /app/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php on line 113, but that's too long for the title.
I'm trying to set up a Symfony project locally, but I'm struggling to get the database connection to work. My parameters.yml looks as follows
parameters:
database_host: 127.0.0.1
database_port: 3306
database_name: database_name
database_user: username
database_password: password
I've been googling this issue a lot and most people seem to solve the issue by changing database_host from localhost to 127.0.0.1, but this doesn't work for me. The app itself runs via Docker, but I've set up the database connection once via Brew and once with a MySQL server for Mac. In both cases I can connect via the command line and with SequelPro/TablePlus, but whenever I try to access the website through the browser I get the "No such file or directory" error.
I've also tried multiple ways of setting up a Docker MySQL container, but can't get it to work. My docker-compose.yml looks like this;
nginx:
build: nginx
ports:
- "8080:80"
links:
- php
volumes:
- ../:/app
php:
build: php-fpm
volumes:
- ../:/app
working_dir: /app
extra_hosts:
- "site.dev: 172.17.0.1"
services:
db:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: 'database_name'
MYSQL_USER: 'username'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password_root'
ports:
- '3306:3306'
expose:
- '3306'
volumes:
- my-db:/var/lib/mysql
But whenever I run docker-compose up -d I get the error Unsupported config option for services: 'db'.
Another attempt was adding
mysql:
image: mysql:latest
volumes:
- mysql_data:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD='password'
- MYSQL_DATABASE='database_name'
- MYSQL_USER='username'
- MYSQL_PASSWORD='password'
To the docker-compose file, and while it does build the mysql image, I can't seem to connect to it with SequelPro/TablePlus. I ran docker-inspect on the container to get the IP (172.17.0.3), but can't seem to get access to it. I can exec into it, login using mysql -u root and create the required user and database, but then I'm still struggling to actually connect to it.
Running docker ps does show the sql container running btw;
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b6de6030791d docker_nginx "nginx -g 'daemon of…" 19 minutes ago Up 14 minutes 0.0.0.0:8080->80/tcp docker_nginx_1
f26b832bb005 docker_php "docker-php-entrypoi…" 19 minutes ago Up 14 minutes 9000/tcp docker_php_1
6c2a9e657435 mysql:latest "docker-entrypoint.s…" 19 minutes ago Up 14 minutes 3306/tcp, 33060/tcp docker_mysql_1
I also thought it might be an issue with changes to the parameters.yml file not properly syncing with the container as I'm using Mac (at my old workplace we had to use docker-sync to make sync changes between our dev environment and the actual container), but when inspecting the container itself using exec I can see the changes in the parameters.yml file.
Could the issue be it trying to connect to a mysql server running outside the Docker container? I'm still very new to Docker so I wouldn't be surprised if that's the mistake. Any tips are appreciated 'cause I'm at a dead end.
Your docker-compose file looks wrong to me, try below docker-compose file.
I removed the links, network is much easier.
version: '3'
services:
nginx:
build: nginx
ports:
- "8080:80"
networks:
- backend
volumes:
- ../:/app
php:
build: php-fpm
volumes:
- ../:/app
working_dir: /app
networks:
- backend
extra_hosts:
- "site.dev: 172.17.0.1"
db:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: 'database_name'
MYSQL_USER: 'username'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password_root'
networks:
- backend
ports:
- '3306:3306'
volumes:
- ./my-db:/var/lib/mysql
networks:
backend:
driver: bridge
then use database_host: db in php file.
I would diagnose
Check docker logs in the mysql container => no errors
Login to the mysql container and login to mysql => no errors
Login to mysql from the host (mysql -u username -p since you are mapping to 3306 port of the host)
Make sure mysql.cnf doesn't block connect from outside(check
bind-address in the mysql configuration if it 127.0.0.1 the its only
allow to connect form locally, i would for now make it 0.0.0.0 or
commented that line if exists)
mysqld --verbose --help => you will see all options
mysqld --verbose --help | grep bind-address=> check the bind-address
Make sure the user i tried to login has enough privileges to
connect(SELECT user,host FROM mysql.user;) check your user can
connect from docker network => 172.* or anywhere=> %
I think your issue is with your parameters.yml:
parameters:
database_host: 127.0.0.1
When you run compose, MySQL and PHP will run in their own containers which will have their own IPs: 127.0.0.1 or localhost from the php won't be able to connect to the db container. It's like you deployed PHP on a virtual machine A and MySQL to another virtual machine B, but you try to access MySQL from machine A by using localhost where you should specify machine B IP or hostname.
With Docker Compose the internal DNS will resolve the service name to it's container, so you can use something like:
parameters:
# name of the service in compose should be resolved
database_host: db
The error SQLSTATE[HY000] [2002] No such file or directory may be caused when the client tries to read MySQL socket usually present at /var/lib/mysql/mysql.sock which is probably not present in your PHP container.
This is strange. I'm currently using Rails 5.1.5 with Docker and Docker-Compose. I am connecting to a remote MySQL (which is firewalled, and has limited access to. No, the database is not inside a docker container; it runs in its own server). I was able to run rails db:migrate and the schema was successfully created.
But, when I try to navigate to the part of the site which has a database call, it displays:
We're sorry, but something went wrong.
I went ahead and enabled STOUT logs to check for everything that was happening. It seems that there is a part in which it says:
Mysql2::Error (Unknown MySQL server host 'db'. (-2));
Note that 'db' is the host for my development environment. The production environment is another one.
I don't think this is a Docker problem (although I could be wrong)
This is the current database.yml:
default: &default
adapter: mysql2
pool: 5
encoding: utf8
database: <%= Rails.application.secrets.mysql_database %>
username: <%= Rails.application.secrets.mysql_username %>
password: <%= Rails.application.secrets.mysql_password %>
host: <%= Rails.application.secrets.mysql_host %>
port: 3306
development: *default
test:
<<: *default
database: db/test.sqlite3
production: *default
The current secrets.yml is as follows:
development:
secret_key_base: the_secret_key_base
mysql_database: <%= ENV["SECRET_MYSQL_DATABASE"] %>
mysql_username: <%= ENV["SECRET_MYSQL_USERNAME"] %>
mysql_password: <%= ENV["SECRET_MYSQL_PASSWORD"] %>
mysql_host: <%= ENV['SECRET_MYSQL_HOST'] %>
I am currently using
config.read_encrypted_secrets = true
And the encrypted secrets.yml.enc is:
This is the Docker-Compose file I'm currently using:
version: '3.2'
services:
app:
image: jja3324/ntid:cprintservicehub_app
restart: always
environment:
RAILS_ENV: production
# What this is going to do is that all the logging is going to be printed into the console.
# Use this with caution as it can become very verbose and hard to read.
# This can then be read by using docker-compose logs app.
RAILS_LOG_TO_STDOUT: 'true'
# The first command, the remove part, what it does is that it eliminates a file that
# tells rails and puma that an instance is running. This was causing issues,
# https://github.com/docker/compose/issues/1393
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -e production -p 5000 -b '0.0.0.0'"
volumes:
- /var/www/cprint
ports:
- "5000:5000"
expose:
- "5000"
# Uses Nginx as a web server (Access everything through http://localhost)
# https://stackoverflow.com/questions/30652299/having-docker-access-external-files
web:
image: jja3324/ntid:cprintservicehub_web
restart: always
links:
- app
volumes:
- type: bind
source: /path-to/ssl/certs
target: /path-to/ssl/certs
- type: bind
source: /path-to-private-ssl/private/
target: /path-to-private-ssl/private
links:
- app
ports:
- "80:80"
- "443:443"
Reading this answer tells me that Rails couldn't resolve the name for the MySQL server. I think this translates to Rails defaulting back to its original config.
Any ideas? Thanks :)
While I haven't solved entirely the problem (Can't connect to the Database yet) it seems this had something to do with Nginx, and config.force_sslin production.rb
Apparently, I had a bug in the Nginx configuration. I was missing setting the X-Forwarded-Proto https header in the configuration file. This was causing infinite redirects (which I honestly don't know why they weren't present the day before... I think it was because of the cookies in my browser).
After doing that, Rails is correctly using my configuration. I still have to figure out what's the problem (which seems to be a firewall issue).