Connect Rails/Unicorn/Nginx container to MySQL container - mysql

Related to this thread, I am trying to create 2 containers: 1 with a rails app, and the other with a MySQL database but I keep getting the Mysql2::Error (Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' in my apps production.log file after I hit the container's IP http://192.168.59.103
When I start the rails container, I am attempting to link them and do get an error if I specify an incorrect MySQL name. What am I missing to successfully link the containers so the full app runs in containers?
Rails container command
docker run --name games-app --link test-mysql:mysql -p 8080 -d -e SECRET_KEY_BASE=test sample_rails_games_app
Here are my files:
Dockerfile
# Publish port 8080
EXPOSE 8080
CMD ["bundle", "exec","unicorn", "-p", "8080"]
CMD ["bunde", "exec", "rake", "db:migrate"]
Rails database.yml (dev and test are the same as production)
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: root
host: localhost
#socket: /tmp/mysql.sock
production:
<<: *default
database: weblog_production
7/31/15 Edit
The docker log shows the unicorn server running:
docker logs a13bf7851c6d
I, [2015-07-31T18:10:59.860203 #1] INFO -- : listening on addr=0.0.0.0:8080 fd=9
I, [2015-07-31T18:10:59.860583 #1] INFO -- : worker=0 spawning...
I, [2015-07-31T18:10:59.864143 #1] INFO -- : master process ready
I, [2015-07-31T18:10:59.864859 #7] INFO -- : worker=0 spawned pid=7
I, [2015-07-31T18:10:59.865097 #7] INFO -- : Refreshing Gem list
I, [2015-07-31T18:11:01.796690 #7] INFO -- : worker=0 ready
7/31/15 Solution Thanks to #Rico
db:migrate was having problems running so I ultimately ran it by hand in a docker run command. Make sure you do this after the container is already created, or during the creation process as it needs the linking to the DB container
This linking article helped me understand that my link was not being created so there was no way to communicate properly.
Once I understood how to accurately make the link, I did update my database.yml with the host and port values
Use this command to check the names of your env variables docker run --rm --name <unique-value> --link <db-name> <non-db-image> env.
Use this to see the value of the links in your app container docker inspect -f "{{ .HostConfig.Links }}" <app-name>

Actually your bundle exec unicorn -p 8080 CMD is superseding the bundle exec rake db:migrate as it doesn't return.
You should run your db:migrate first and you should run it with the RUN command as CMD is the primary command in docker.
But the other problem is with your database.yml file. You are pointing your db to a db server that runs on the same container as in the application. You should populate the values of your database.yml from the env variables created after you link your source container (application) to destination container (db server container). The env variables are created in the source container.
More info here: https://docs.docker.com/userguide/dockerlinks/
So for example:
$ docker run --rm --name web2 --link db:db training/webapp env
. . .
DB_NAME=/web2/db
DB_PORT=tcp://172.17.0.5:5432
DB_PORT_5432_TCP=tcp://172.17.0.5:5432
DB_PORT_5432_TCP_PROTO=tcp
DB_PORT_5432_TCP_PORT=5432
DB_PORT_5432_TCP_ADDR=172.17.0.5
Your database.yml should look something like this:
default: &default
adapter: mysql2
encoding: utf8
pool: 5
database: <%= ENV['DB_NAME'] %>
username: root
password: root
host: <%= ENV['DB_PORT_5432_TCP_ADDR'] %>
port: <%= ENV['DB_PORT_5432_TCP_PORT'] %>

You can't have 2 CMD commands in your Dockerfile, in fact only the last one is kept. The CMD command executed is `
CMD ["bunde", "exec", "rake", "db:migrate"]`
the other, the
CMD ["bundle", "exec","unicorn", "-p", "8080"]
is superseded.
See Supervisor
https://docs.docker.com/articles/using_supervisord/
if you want to run more than one rocess in your container, or run 2 differnet containers

Related

Cannot connect to local MySQL server using socket (2) in Elastic beanstalk using RDS database

I am creating a CI\CD for my Ruby on Rails application running in Elastic Beanstalk using code pipeline. My Code build succeeds but deployment to EBS provider fails with the below error
+ cd /var/app/ondeck
+ su -s /bin/bash -c 'bundle exec /opt/elasticbeanstalk/support/scripts/check-for-rake-task.rb db:migrate' webapp
`/home/webapp` is not a directory.
Bundler will use `/tmp/bundler20211229-11836-pzl6dq11836' as your home directory temporarily.
+ '[' false = true ']'
+ su -s /bin/bash -c 'leader_only bundle exec rake db:migrate' webapp
`/home/webapp` is not a directory.
Bundler will use `/tmp/bundler20211229-11840-9hxmcz11840' as your home directory temporarily.
rake aborted!
Mysql2::Error::ConnectionError: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
/opt/rubies/ruby-2.6.7/bin/bundle:23:in `load'
/opt/rubies/ruby-2.6.7/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace) (Executor::NonZeroExitStatus)
I have
mysql --version
mysql Ver 14.14 Distrib 5.5.62, for Linux (x86_64) using readline 5.1
and could connect to External RDS MySQL Aurora database.
my Buildspec.yml:
version: 0.2
phases:
install:
runtime-versions:
ruby: 2.6
commands:
- echo Installing Bundler...
- gem install bundler
- bundle install
pre_build:
commands:
- echo Preparing database to run tests...
- bundle exec rails db:create --trace RAILS_ENV=production
build:
commands:
- echo Running tests...
- bundle exec rails test/test_helper.rb
artifacts:
files:
- '**/*'
My database.yml:
default: &default
adapter: mysql2
encoding: utf8mb4
collation: utf8mb4_unicode_ci
pool: 5
timeout: 5000
username: <%= ENV['DB_USER'] %>
password: <%= ENV['DB_PASSWORD'] %>
host: <%= ENV['DB_HOST'] %>
port: <%= ENV['DB_PORT'] %>
development:
<<: *default
database: <%= ENV['DB'] %>
test:
<<: *default
database: <%= ENV['DB_TEST'] %>
production:
<<: *default
database: <%= ENV['DB'] %>`
I found an answer for a similar problem connecting to a mysql server from Rails EB SSH that might still help you. Basically, Elastic Beanstalk was not successfully using my environment's envars to run my bundle exec rails console command in the eb ssh instance access.
I solved my issue by prepending all of the required envars explicitly to any rails commands I ran from within the eb ssh instance access. So for example, in order to run your su -s /bin/bash -c 'leader_only bundle exec rake db:migrate' webapp command, you may have to change that to run something like the following:
su -s /bin/bash -c 'leader_only RAILS_MASTER_KEY=xxxxxxx RAILS_ENV=production DB_HOST=xxxxxxx DB_PASSWORD=xxxxxxx DB_USER=xxxxxxx DB=xxxxxx AWS_REGION=xxxxxxx AWS_BUCKET=xxxxxxx bundle exec rake db:migrate' webapp
Replace the xxxxxxxs above with the values from the corresponding variables in your EB > Configuration > Software tab, and you should be able to connect to the remote database that way. After that, you will be able to run migration commands, rake tasks and other database-reliant functions. I'm not familiar with CI/CD but I hope the principles of this solution are adaptable to your project.
Thanks for your answers. But I faced the issue because of the master branch in git not updated properly.
After updating the master branch with proper application code, the CI\CD initiated as expected but got stuck in between because of env and credential file not in git.
I create a s3 bucket to store the env files and referred those file while packaging it for EBS deployment.
Hope this helps someone in future :)

Rails class method in cron job not working in elastic beanstalk

I have a class method that I need to run every 15 min, I have the cron job
0,15,30,45 * * * * /bin/bash -l -c 'cd /var/app/current && sudo /opt/rubies/ruby-2.3.5/bin/bundle exec /opt/rubies/ruby-2.3.5/bin/rails runner -e production '\''Structure.check_parking'\'' >> /var/app/current/log/cron_log 2>&1'
Running in my elastic beanstsalk environment, but I keep getting an error of
/opt/rubies/ruby-2.3.5/lib/ruby/gems/2.3.0/gems/mysql2-0.4.4/lib/mysql2/client.rb:87:in `connect': Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
I am using an RDS server in my production environment so it is strange that it's trying to connect to a local mysql server.
The rails app is connected to the database and I can do all normal functionality on it but when I try to run this class method I am getting this error.
My guess is that it's not running in production, but I'm really not sure, my database.yml file looks like
development:
adapter: mysql2
database: bddatabase
encoding: utf8
username: bduser
password: dbpass
host: 127.0.0.1
port: 3306
production:
adapter: mysql2
encoding: utf8
database: <%= ENV['RDS_DB_NAME'] %>
username: <%= ENV['RDS_USERNAME'] %>
password: <%= ENV['RDS_PASSWORD'] %>
host: <%= ENV['RDS_HOSTNAME'] %>
port: <%= ENV['RDS_PORT'] %>
Been trying to figure this out for a while now, any help is really appreciated!
I figured it out, turns out there was a problem with my cron task it's self. I think how I was referencing bundle and rails the way I did it wasn't pointing at my app which was causing the error. Changing the command to
0,15,30,45 * * * * /bin/bash -l -c 'cd /var/app/current && bundle exec rails runner -e production 'Structure.check_parking' >> /var/app/current/log/cron_log 2>&1'
Did the trick

AWS Elastic Beanstock. rails cannot connect to MYSQL

I cannot solve this issue, mysql will not connect. using eb deploy. And I am using Ruby v 2.3.1
+ su -s /bin/bash -c 'leader_only bundle exec rake db:migrate' webapp
rake aborted! Mysql2::Error: Can't connect to MySQL server on 'labr.cyp2t1256323.us-west-2.rds.amazonaws.com' (4)
/opt/rubies/ruby-2.3.1/bin/bundle:23:in `load'
/opt/rubies/ruby-2.3.1/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace) (ElasticBeanstalk::ExternalInvocationError)
caused by: ++ /opt/elasticbeanstalk/bin/get-config container -k script_dir
Database.yml
production:
adapter: mysql2
encoding: utf8
reconnect: false
pool: 20
database: ***
username: *****
password: *****
host: labr.cyp2t1256323.us-west-2.rds.amazonaws.com
port: 3306
3306 is enabled on Amazon EC2 console, rails-env is at health red
Add a new security group that accept all traffic. And add your db instance to this group.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html

How to configure circle.yml in order to call the database?

I am trying to configure the CircleCi setup.
When I try to run the tests in my own environment then everything works out, but when I try to run it in Circle CI, then I haave got this error.
PHPUnit 5.6.0 by Sebastian Bergmann and contributors.
IIIII.IIIIIIIIIIIIIIIIIIIIIIIIIIIIIFFFFFFFFFFFFIIII..IIIIIIIIIIII 65 / 89 ( 73%)
IIIIIIIIIIIIIIIIIIIIIII. 89 / 89 (100%)
Time: 1.27 seconds, Memory: 56.00MB
There were 12 failures:
1) App\Test\TestCase\Controller\TagsControllerTest::testStages
exception 'PDOException' with message 'SQLSTATE[HY000] [2002] No such file or directory' in /home/ubuntu/cemcloud2/vendor/cakephp/cakephp/src/Database/Driver/PDODriverTrait.php:48
This is the first part of the error.
I am getting that I have an error with my database configuration.
We are using Vagrant with the script (not sure if this is viable).
Also we are using mariaDB 10.1.19 and I have built up a test database for PHPUnit to run the tests against.
I don't know how to add this database with the configuration of mariadb and specific database.
I know that I have to add the
database.yml.ci
to my config file so I have added it in there and it looks like this.
test:
adapter: mysql
database: cemcloudTest
username: root
password: ''
Also I have added the circle.yml to our project root and it looks like this.
machine:
php:
version: 7.0
mysql:
version: 10.1.19
database:
override:
- cp config/database.yml.ci config/database.yml
- bundle exec rake db:create db:schema:load
Could someone help me out ?
in your cicle.yml database have to be created with name you use in your test config.
database:
override:
- mysql -u ubuntu -e "create database circle_ruby_test"
So that, this is the db conf:
test: # connection name
adapter: mysql2
encoding: utf8
reconnect: false
host: localhost
database: circle_ruby_test #your created DB
username: ubuntu #default circle user
pool: 25
socket: /var/run/mysqld/mysqld.sock # default sock location for circle
HTH:)

Create a new Ruby on Rails application using MySQL instead of SQLite

I want to create my Rails application with MySQL, because I like it so much. How can I do that in the latest version of Rails instead of the default SQLite?
Normally, you would create a new Rails app using
rails ProjectName
To use MySQL, use
rails new ProjectName -d mysql
If you already have a rails project, change the adapter in the config/database.yml file to mysql and make sure you specify a valid username and password, and optionally, a socket:
development:
adapter: mysql2
database: db_name_dev
username: koploper
password:
host: localhost
socket: /tmp/mysql.sock
Next, make sure you edit your Gemfile to include the mysql2 or activerecord-jdbcmysql-adapter (if using jruby).
For Rails 3 you can use this command to create a new project using mysql:
$ rails new projectname -d mysql
Go to the terminal and write:
rails new <project_name> -d mysql
If you have not created your app yet, just go to cmd(for windows) or terminal(for linux/unix) and type the following command to create a rails application with mysql database:
$rails new <your_app_name> -d mysql
It works for anything above rails version 3. If you have already created your app, then you can do one of the 2 following things:
Create a another_name app with mysql database, go to cd another_name/config/ and copy the database.yml file from this new app. Paste it into the database.yml of your_app_name app. But ensure to change the database names and set username/password of your database accordingly in the database.yml file after doing so.
OR
Go to cd your_app_name/config/ and open database.yml. Rename as following:
development:
adapter: mysql2
database: db_name_name
username: root
password:
host: localhost
socket: /tmp/mysql.sock
Moreover, remove gem 'sqlite3' from your Gemfile and add the gem 'mysql2'
If you are using rails 3 or greater version
rails new your_project_name -d mysql
if you have earlier version
rails new -d mysql your_project_name
So before you create your project you need to find the rails version. that you can find by
rails -v
rails -d mysql ProjectName
rails new <project_name> -d mysql
OR
rails new projectname
Changes in config/database.yml
development:
adapter: mysql2
database: db_name_name
username: root
password:
host: localhost
socket: /tmp/mysql.sock
Create application with -d option
rails new AppName -d mysql
$ rails --help
is always your best friend
usage:
$ rails new APP_PATH[options]
also note that options should be given after the application name
rails and mysql
$ rails new project_name -d mysql
rails and postgresql
$ rails new project_name -d postgresql
You should use the switch -D instead of -d because it will generate two apps and mysql with no documentation folders.
rails -D mysql project_name (less than version 3)
rails new project_name -D mysql (version 3 and up)
Alternatively you just use the --database option.
In Rails 3, you could do
$rails new projectname --database=mysql
If you are creating a new rails application you can set the database using the -d switch like this:
rails -d mysql myapp
Its always easy to switch your database later though, and using sqlite really is easier if you are developing on a Mac.
Just go to rails console and type:
rails new YOURAPPNAME -d mysql
On new project, easy peasy:
rails new your_new_project_name -d mysql
On existing project, definitely trickier. This has given me a number of issues on existing rails projects. This kind of works with me:
# On Gemfile:
gem 'mysql2', '>= 0.3.18', '< 0.5' # copied from a new project for rails 5.1 :)
gem 'activerecord-mysql-adapter' # needed for mysql..
# On Dockerfile or on CLI:
sudo apt-get install -y mysql-client libmysqlclient-dev
First make sure that mysql gem is installed, if not? than type following command in your console
gem install mysql2
Than create new rails app and set mysql database as default database by typing following command in your console
rails new app-name -d mysql
Use following command to create new app for API with mysql database
rails new <appname> --api -d mysql
adapter: mysql2
encoding: utf8
pool: 5
username: root
password:
socket: /var/run/mysqld/mysqld.sock
database.yml
# MySQL. Versions 5.1.10 and up are supported.
#
# Install the MySQL driver
# gem install mysql2
#
# Ensure the MySQL gem is defined in your Gemfile
# gem 'mysql2'
#
# And be sure to use new-style password hashing:
# https://dev.mysql.com/doc/refman/5.7/en/password-hashing.html
#
default: &default
adapter: mysql2
encoding: utf8
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
host: localhost
database: database_name
username: username
password: secret
development:
<<: *default
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
# As with config/secrets.yml, you never want to store sensitive information,
# like your database password, in your source code. If your source code is
# ever seen by anyone, they now have access to your database.
#
# Instead, provide the password as a unix environment variable when you boot
# the app. Read http://guides.rubyonrails.org/configuring.html#configuring-a-database
# for a full rundown on how to provide these environment variables in a
# production deployment.
#
# On Heroku and other platform providers, you may have a full connection URL
# available as an environment variable. For example:
#
# DATABASE_URL="mysql2://myuser:mypass#localhost/somedatabase"
#
# You can use this database configuration with:
#
# production:
# url: <%= ENV['DATABASE_URL'] %>
#
production:
<<: *default
Gemfile:
# Use mysql as the database for Active Record
gem 'mysql2', '>= 0.4.4', '< 0.6.0'
you first should make sure that MySQL driver is on your system if not run this on your terminal if you are using Ubuntu or any Debian distro
sudo apt-get install mysql-client libmysqlclient-dev
and add this to your Gemfile
gem 'mysql2', '~> 0.3.16'
then run in your root directory of the project
bundle install
after that you can add the mysql config to config/database.yml as the previous answers