Unknown MySQL server host 'db' Rails and Docker - mysql

This is strange. I'm currently using Rails 5.1.5 with Docker and Docker-Compose. I am connecting to a remote MySQL (which is firewalled, and has limited access to. No, the database is not inside a docker container; it runs in its own server). I was able to run rails db:migrate and the schema was successfully created.
But, when I try to navigate to the part of the site which has a database call, it displays:
We're sorry, but something went wrong.
I went ahead and enabled STOUT logs to check for everything that was happening. It seems that there is a part in which it says:
Mysql2::Error (Unknown MySQL server host 'db'. (-2));
Note that 'db' is the host for my development environment. The production environment is another one.
I don't think this is a Docker problem (although I could be wrong)
This is the current database.yml:
default: &default
adapter: mysql2
pool: 5
encoding: utf8
database: <%= Rails.application.secrets.mysql_database %>
username: <%= Rails.application.secrets.mysql_username %>
password: <%= Rails.application.secrets.mysql_password %>
host: <%= Rails.application.secrets.mysql_host %>
port: 3306
development: *default
test:
<<: *default
database: db/test.sqlite3
production: *default
The current secrets.yml is as follows:
development:
secret_key_base: the_secret_key_base
mysql_database: <%= ENV["SECRET_MYSQL_DATABASE"] %>
mysql_username: <%= ENV["SECRET_MYSQL_USERNAME"] %>
mysql_password: <%= ENV["SECRET_MYSQL_PASSWORD"] %>
mysql_host: <%= ENV['SECRET_MYSQL_HOST'] %>
I am currently using
config.read_encrypted_secrets = true
And the encrypted secrets.yml.enc is:
This is the Docker-Compose file I'm currently using:
version: '3.2'
services:
app:
image: jja3324/ntid:cprintservicehub_app
restart: always
environment:
RAILS_ENV: production
# What this is going to do is that all the logging is going to be printed into the console.
# Use this with caution as it can become very verbose and hard to read.
# This can then be read by using docker-compose logs app.
RAILS_LOG_TO_STDOUT: 'true'
# The first command, the remove part, what it does is that it eliminates a file that
# tells rails and puma that an instance is running. This was causing issues,
# https://github.com/docker/compose/issues/1393
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -e production -p 5000 -b '0.0.0.0'"
volumes:
- /var/www/cprint
ports:
- "5000:5000"
expose:
- "5000"
# Uses Nginx as a web server (Access everything through http://localhost)
# https://stackoverflow.com/questions/30652299/having-docker-access-external-files
web:
image: jja3324/ntid:cprintservicehub_web
restart: always
links:
- app
volumes:
- type: bind
source: /path-to/ssl/certs
target: /path-to/ssl/certs
- type: bind
source: /path-to-private-ssl/private/
target: /path-to-private-ssl/private
links:
- app
ports:
- "80:80"
- "443:443"
Reading this answer tells me that Rails couldn't resolve the name for the MySQL server. I think this translates to Rails defaulting back to its original config.
Any ideas? Thanks :)

While I haven't solved entirely the problem (Can't connect to the Database yet) it seems this had something to do with Nginx, and config.force_sslin production.rb
Apparently, I had a bug in the Nginx configuration. I was missing setting the X-Forwarded-Proto https header in the configuration file. This was causing infinite redirects (which I honestly don't know why they weren't present the day before... I think it was because of the cookies in my browser).
After doing that, Rails is correctly using my configuration. I still have to figure out what's the problem (which seems to be a firewall issue).

Related

Is there a way to use a generic hostname that will work in both Docker and localhost?

I’m on Mac OS Big Sur and running the following Docker versions
$ docker -v
Docker version 20.10.12, build e91ed57
$ docker-compose -v
Docker Compose version v2.2.3
I’m running a Rails 6 app, whose database is configured like so
development:
adapter: mysql2
encoding: utf8
host: host.docker.internal
database: cfs
pool: 5
username: myuser
password: bypass
I have to use “host.docker.internal” so that the Rails app can access the Docker db when running inside of Docker, set up in my docker-compose.yml like so
services:
db:
image: mysql:5.7
command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
restart: always
volumes:
- ./docker/provision/mysql/init:/docker-entrypoint-initdb.d
environment:
MYSQL_ROOT_PASSWORD: bypass
ports:
# <Port exposed> : < MySQL Port running inside container>
- '3306:3306'
expose:
# Opens port 3306 on the container
- '3306'
…
web:
#restart: always
build: ./my-project
ports:
- "3000:3000"
expose:
- '3000'
command: foreman start
volumes:
- ./my-project/:/app
depends_on:
- db
However if I run the app locally without Docker, I have to change my config files to remove “host.docker.internal” and use “127.0.0.1” instead (or localhost).
Is there a way I can set things up so that I have a single database config file that works in both Docker and without Docker such that I don’t have to change the host around?
You can use ERB markup in the database.yml file, which lets you use an environment variable here. In general I'd suggest making default values for things be whatever default will work in a plain non-Docker development environment.
# config/database.yml
development:
host: <%= ENV['DATABASE_HOST'] || 'localhost' %>
If you're running this in a setup where the database hostname isn't localhost – it's in a sibling container, or you're using a cloud-hosted database like Amazon RDS – you can set that environment variable.
# docker-compose.yml
version: '3.8'
services:
db:
image: 'mysql:5.7'
et: cetera
web:
build: .
ports:
- '3000:3000'
depends_on:
- db
environment:
DATABASE_HOST: db # <-- will be read in database.yml

Docker-Rails Can't Connect localhost mysql database

I'm trying to have a docker container use my local mysql database. Using docker-compose up, I am able to get all the pieces running, however, the db exits with status code 1, and the rails application cannot connect to any database instance. I know that mysql is running on my system and that the service has been started. My docker-compose file:
version: '3'
services:
db:
image: mysql
restart: always
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: db
MYSQL_USER: root
MYSQL_PASSWORD: password
volumes:
- ./tmp/db:/var/lib/mysql
web:
build: .
image: rails-devise:1.0.0
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- ".:/baas-status"
ports:
- "3000:3000"
depends_on:
- db
I know the containers are running because docker ps gives the following output:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a49aba0cd68 rails-devise:1.0.0 "entrypoint.sh bash …" 23 minutes ago Up 2 minutes 0.0.0.0:3000->3000/tcp status_web_1
38554baf5efc mysql "docker-entrypoint.s…" 23 minutes ago Up 2 minutes 3306/tcp, 0.0.0.0:3002->3002/tcp, 33060/tcp status_db_1
Even though I exposed port 3002 and want to use that as the port for the database, is it unable to connect because I'm on the wrong port? I see in the docker ps output that mysql is on port 3306.
My database.yml file:
base: &base
adapter: mysql2
encoding: utf8
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
default: &default
<<: *base
username: <%= ENV['DB_USERNAME'] %>
password: <%= ENV['DB_PASSWORD'] %>
host: <%= ENV['DB_HOST'] %>
database: <%= ENV['DB_NAME'] %>
development:
<<: *base
username: root
password: password
host: db
database: db
I have also tried editing the host for development to be host: db. That didn't change anything either. I tried running docker-compose run web rake db:create as well, and I get the following error:
This has been fixed with the current changes to the docker-compose file
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
At this point, I'm at a loss on where to proceed. Any information or help would be greatly appreciated. Thank you.
Edit:
I have changed the ports for the db and it seems to have fixed the connection issues. Now I'm running into an error that gives
web_1 | Mysql2::Error::ConnectionError (Plugin caching_sha2_password could not be loaded: /usr/lib/x86_64-linux-gnu/mariadb19/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory):
I'm going to do some research to figure out the meaning behind this error, any tips to move in the right direction would be greatly appreciated though!
Edit2:
I ended up connecting the db by reverting mysql back to 5.7.18 to avoid the sha authentication. The docker-compose file and settings that worked for me are as follows:
docker-compose.yml
version: '2'
services:
db:
image: mysql:5.7.18
restart: always
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=root
volumes:
- db-data:/var/lib/mysql
web:
build: .
image: rails-devise:1.0.0
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- ".:/baas-status"
ports:
- 3000:3000
links:
- db
depends_on:
- db
volumes:
db-data:
driver: local
The database.yml file remained the same with the exception of changing the password to root. This is the solution that worked for me.
Resource I ended up using!
Sounds like you figured out the connection issue. You'll want to connect on 3306, not 3002 based on the docker-compose file you posted.
I assume you also made sure to define these environment variables in your rails docker container as well. Is that correct?
username: <%= ENV['DB_USERNAME'] %>
password: <%= ENV['DB_PASSWORD'] %>
host: <%= ENV['DB_HOST'] %>
database: <%= ENV['DB_NAME'] %>
Since you do not define a tag of the mysql image to use in your docker-compose file, it will pull the latest image which is mysql ~8.x. MySQL 8 uses the caching_sha2_password authentication plugin by default (unless you specify otherwise). The Rails mysql2 driver version you're using likely does not support this authentication mechanism yet.
So, if you would like to revert to an older authentication plugin (like the one from mysql 5.7), you can use a command line flag in your mysql container like so:
services:
...
db:
image: mysql:8.0
...
command: mysqld --default-authentication-plugin=mysql_native_password
More information can also be found on this SO answer.

I want to delete "Mysql2 :: Error :: ConnectionError (Unknown MySQL server host 'db' (0)):"

environment
- MacOS Mojave 10.14.6
- Ruby 2.6.4
- Rails 5.2.2
- mysql Ver 8.0.17 for osx10.14 on x86_64 (Homebrew)
- Docker 2.1.0.3
What I want to realize
I want to clear the following error.
Mysql2::Error::ConnectionError (Unknown MySQL server host 'db' (0)):
Terminal
$ rails s
=> Booting Puma
=> Rails 5.2.2 application starting in development
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.12.1 (ruby 2.6.4-p104), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://localhost:3000
Use Ctrl-C to stop
Started GET "/" for ::1 at 2019-10-07 22:11:28 +0900
Mysql2::Error::ConnectionError (Unknown MySQL server host 'db' (0)):
...
$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------
rails_test_db_1 docker-entrypoint.sh mysqld Up 3306/tcp, 33060/tcp
rails_test_web_1 bundle exec rails s -p 300 ... Exit 1
docker-compose.yml
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- 3000:3000
depends_on:
- db
tty: true
stdin_open: true
db:
image: mysql:5.7
volumes:
- db-volume:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: password
volumes:
db-volume:
database.yml
# MySQL. Versions 5.0 and up are supported.
#
# Install the MySQL driver
# gem install mysql2
#
# Ensure the MySQL gem is defined in your Gemfile
# gem 'mysql2'
#
# And be sure to use new-style password hashing:
# http://dev.mysql.com/doc/refman/5.7/en/old-client.html
#
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: password
host: db
development:
<<: *default
database: app_development
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: app_test
# As with config/secrets.yml, you never want to store sensitive information,
# like your database password, in your source code. If your source code is
# ever seen by anyone, they now have access to your database.
#
# Instead, provide the password as a unix environment variable when you boot
# the app. Read http://guides.rubyonrails.org/configuring.html#configuring-a-database
# for a full rundown on how to provide these environment variables in a
# production deployment.
#
# On Heroku and other platform providers, you may have a full connection URL
# available as an environment variable. For example:
#
# DATABASE_URL="mysql2://myuser:mypass#localhost/somedatabase"
#
# You can use this database configuration with:
#
# production:
# url: <%= ENV['DATABASE_URL'] %>
#
production:
<<: *default
database: app_production
username: app
password: <%= ENV['APP_DATABASE_PASSWORD'] %>
That error is telling you that the host db is unknown.
You are running rails s on your terminal. That is starting your rails server outside of docker, therefore the host db cannot be resolved. (You only have that kind of DNS resolution for the containers running within the same docker network)
You should just run docker-compose up so that both services defined in your docker-compose.yml (web and db) start in the same network (docker-compose takes care of that, you don't need to do any extra setup for it)
You can create a network that will be shared inside your docker between images
networks:
my_network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.25.0.1/16
gateway: 172.25.0.1
Then assign specific IP to your db container:
networks:
my_network:
ipv4_address: 172.25.0.3
After that you need to make sure db hostname is linked with db container IP.
The code bellow will add new line 172.25.0.3 db to you /etc/hosts file.
extra_hosts:
- 'db:172.25.0.3'
Final version of docker-compose will look like this:
version: '3'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- 3000:3000
depends_on:
- db
tty: true
stdin_open: true
extra_hosts:
- 'db:172.25.0.3'
db:
image: mysql:5.7
volumes:
- db-volume:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: password
networks:
my_network:
ipv4_address: 172.25.0.3
volumes:
db-volume:
networks:
my_network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.25.0.1/16
gateway: 172.25.0.1
If you do this you can be sure that db host will be always available. I hope that will help you.

Docker + Rails + MySQL = Ignoring environment variables in database.yml

I'm newbie using docker and my issue is to load a Rails 4 application using MySQL and Docker with environment variables.
Basically, I build the app and run it. In another terminal I create mysql database and user/password to be used by the database.yml. Here are the steps that I run:
$ docker-compose build
$ docker-compose up -d
$ docker exec -it poseidon_db_1 bash -l
#root#dockerimg mysql -uroot -proot
mysql> create database poseidon_development;
mysql> CREATE USER 'poseidon_user'#'localhost' IDENTIFIED BY 'poseidon_password';
mysql> CREATE USER 'poseidon_user'#'%' IDENTIFIED BY 'poseidon_password';
mysql> GRANT ALL ON poseidon_development.* TO 'poseidon_user'#'localhost';
mysql> GRANT ALL ON poseidon_development.* TO 'poseidon_user'#'%';
mysql> FLUSH PRIVILEGES;
mysql> exit
#root#dockerimg exit
$ docker-compose run web bundle exec rake db:migrate
database.yml
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: <%= ENV["DATABASE_USERNAME"] %>
password: <%= ENV["DATABASE_PASSWORD"] %>
database: <%= ENV["DATABASE_NAME"] %>
host: db
development:
<<: *default
test:
<<: *default
docker-compose.yml
version: '3.6'
services:
db:
restart: always
image: mysql:5.6.34
ports:
- "3306:3306"
volumes:
- dbdata
env_file:
- .env
dbdata:
image: tianon/true
volumes:
- /var/lib/mysql
web:
build: .
ports:
- '3000:3000'
volumes:
- webdata
links:
- "db"
env_file:
- .env
webdata:
image: tianon/true
volumes:
- ".:/var/www/poseidon"
Dockerfile
FROM ruby:2.3.7
ENV LANG C.UTF-8
RUN apt-get update -qq
RUN apt-get install -y build-essential mysql-client vim
WORKDIR /tmp
ADD Gemfile Gemfile
ADD Gemfile.lock Gemfile.lock
RUN bundle install
ENV APP_ROOT /var/www/poseidon
RUN mkdir -p $APP_ROOT
WORKDIR $APP_ROOT
COPY . $APP_ROOT
RUN chmod a+x .env
EXPOSE 3000
ENTRYPOINT ["bundle", "exec"]
CMD ["rails", "server", "-p", "3000", "-b", "0.0.0.0"]
.env
SECRET_KEY_BASE=xxxxxxxx
RAILS_ENV=development
RAILS_SERVE_STATIC_FILES=true
URL_OPTIONS=domain.com
RECAPTCHA_SECRET_KEY=xxxxxxx
RECAPTCHA_SITE_KEY=xxxxxxx
MAILGUN_API=xxxxxx
MAILGUN_DOMAIN=xxxxxxx
MYSQL_ROOT_PASSWORD=root
DATABASE_USERNAME=poseidon_user
DATABASE_PASSWORD=poseidon_password
DATABASE_NAME=poseidon_development
The problem is, when accessing http://127.0.0.1, I receive Access denied for user '<%= ENV["DATABASE_USERNAME"] %>'#'IP' (using password: YES)
I don't know what to do anymore, it seems to not load the env vars correctly.
Any help will be really appreciated!
Regards
I was stumped on this issue for a day or two. Things were working fine out of Docker but for some reason within the container the ERB wasn't being parsed.
This post seemed to help: rails database.yml not accepting ERB (which I found via Rails not replace ENV's value in database.yml)
I just experienced the same thing, and came across your post. I had
been following a tutorial that had me create a puma.conf file that
contained the code below:
ActiveRecord::Base.establish_connection( YAML.load_file( "#{app_dir}/config/database.yml" )[rails_env])
I modified to the following, and everything worked as expected:
require 'erb'
ActiveRecord::Base.establish_connection( YAML.load( ERB.new( File.read( "#{app_dir}/config/database.yml" )).result)[rails_env])

Docker MySQL can't connect to socket

I'm learning Docker and I've a problem trying to connect a Rails app on the passenger-full container and a mysql container. Both are linked in a compose file
app:
build: ./rails
ports:
- "80:80"
links:
- database
volumes:
- ./rails:/home/app/webapp
database:
image: mysql
environment:
- MYSQL_DATABASE="dockertest"
- MYSQL_USER="dockertest"
- MYSQL_PASSWORD="dockertest"
- MYSQL_ROOT_PASSWORD="root"
So I added the apt-get install at the top of my Dockerfile like this
FROM phusion/passenger-full
RUN apt-get update && apt-get install libmysqlclient-dev mysql-client -y
# Set correct environment variables.
ENV HOME /root
# Use baseimage-docker's init process.
CMD ["/sbin/my_init"]
RUN rm -f /etc/service/nginx/down
RUN rm /etc/nginx/sites-enabled/default
ADD webapp.conf /etc/nginx/sites-enabled/webapp.conf
RUN mkdir /home/app/webapp
WORKDIR /home/app/webapp
ADD . /home/app/webapp
RUN cd /home/app/webapp && bundle install
RUN touch /home/app/webapp/tmp/restart.txt
# Clean up APT when done.
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Also this is my database.yml in the Rails app.
default: &default
adapter: mysql2
database: dockertest
host: <%= ENV['MYSQL_PORT_3306_TCP_ADDR'] %>
port: <%= ENV['MYSQL_PORT_3306_TCP_PORT'] %>
username: dockertest
password: dockertest
development:
<<: *default
production:
<<: *default
The problem is that I cant stop receiving the error
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
The webconf file is
# /etc/nginx/sites-enabled/webapp.conf:
server {
listen 80;
server_name localhost;
root /home/app/webapp/public;
passenger_enabled on;
passenger_user app;
passenger_ruby /usr/bin/ruby2.2;
}
Is that the right way to do this? As you can see I'm pretty new to docker.
The problem here is with the links directive in your docker-compose.yml file. You have:
links:
- database
That's basically saying that the link name:alias is database:database, according to the docker-compose.yml reference.
Also, if you read the linking container docs you can see that the environments exported to the source container are of the format ALIAS_XXX for example ALIAS_PORT_3306_TCP_PORT. So in essence in your database.yml what you want to do is something like this:
default: &default
adapter: mysql2
database: dockertest
host: <%= ENV['DATABASE_PORT_3306_TCP_ADDR'] %>
port: <%= ENV['DATABASE_PORT_3306_TCP_PORT'] %>
username: dockertest
password: dockertest
development:
<<: *default
production:
<<: *default
If you want to use the MYSQL alias your links would have to look something like this in your docker-compose.yml file.
links:
- database:mysql
The error:
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
is basically coming from your Rails app not being to see what's in your database.yml and defaulting to a local /var/run/mysqld/mysqld.sock connection.
Hope it helps.