I'm having an issue connecting to an external AWS Aurora MySQL in my Rails app in production.
Here is the setup:
One main PostgreSQL database for the app
One external (AWS Aurora MySQL) database used as a reader for huge datasets
In development, everything works fine, but when I deploy to Heroku, I can only successfully query one table of the external database. When I create another table I get this error message:
ActionView::Template::Error (Mysql2::Error::ConnectionError: Lost connection
to MySQL server during query: SELECT `TMC_Identification`.`direction`,
`TMC_Identification`.`miles`, `TMC_Identification`.`road`,
`TMC_Identification`.`tmc` FROM `TMC_Identification`) :
1: <%= raw(#tmcs.to_json) %>
Rails 5.2
Ruby 2.5.3
Models:
class TmcReading < ApplicationRecord
establish_connection(:tmc_data)
self.table_name = "TMC_Readings"
end
class TmcIdentification < ApplicationRecord
establish_connection(:tmc_data)
self.table_name = "TMC_Identification"
end
database.yml:
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
production:
<<: *default
database: production
username: admin
password: <%= ENV['DATABASE_PASSWORD'] %>
tmc_data:
adapter: mysql2
encoding: utf8
database: tmc_data
username: <%= Rails.application.credentials.tmc_data_db[:username] %>
password: <%= Rails.application.credentials.tmc_data_db[:password] %>
host: tmc-data.cluster-ro-xyz.us-east-1.rds.amazonaws.com
port: 3306
Controller Action
def tmc_identifications
#tmcs = TmcIdentification.all.select(:direction, :miles, :road, :tmc)
end
View
<%= raw(#tmcs.to_json) %>
Everything works fine in development, but not in production. The same database and credentials are used in production for the "tmc_data" connection.
I assume I'm having some thread safety issues, but I'm not sure how to fix that.
The way I solved this problem is the following way:
Models:
class TmcData < ActiveRecord::Base
self.abstract_class = true
establish_connection(:tmc_data)
end
class TmcReading < ApplicationRecord
self.table_name = "TMC_Readings"
end
class TmcIdentification < ApplicationRecord
self.table_name = "TMC_Identification"
end
More details: https://www.thegreatcodeadventure.com/managing-multiple-databases-in-a-single-rails-application/
Also, I stopped connecting to the same secondary database with my development and production environment.
I have a rails application (ruby 2.0.0, Rails 4.0.1) that connects to several MySQL database.
The connection to the local database always work properly, this is my configuration:
production:
adapter: mysql
encoding: utf8
database: xxx
username: yyy
password: zzz
host: x.x.x.x
port: 3306
reconnect: true
But connections to remote databases often return an error, such as connecting to my radius external database returns the following error:
Mysql :: Error:: SELECT 1 AS one FROM WHERE `` radacct` radacct`.`username` =? LIMIT 1
Updating the page several times, the connection is restored and I can see the data properly. This is my configuration:
radius:
port: 3306
adapter: mysql
encoding: utf8
database: xxx
username: yyy
password: zzz
host: x.x.x.x
reconnect: true
connect_timeout: 60
read_timeout: 90
write_timeout: 90
checkout_timeout: 10
reaping_frequency: 30
dead_connection_timeout: 30
I tried to change the configuration of the timers in different ways, but the problem persists.
to connect to the radius server I created the following model common to all:
class RadActiveRecord <ActiveRecord :: Base
self.abstract_class = true
establish_connection "radius"
end
for example for the table radacct use the following model:
class RadAcctDetail <RadActiveRecord
self.table_name = "radacct"
end
the error in question is with any queries, such as:
def index
#rad_acct_details = RadAcctDetail.all.paginate = (
: order => "radacctid DESC"
: page => params [: page] || 1
: per_page => params [: per_page] || 25)
end
Does anyone have any suggestions?
I've been building a rails app using a postgres database but of course I also have been tasked with importing data from a legacy mysql database.
The way I've been handling this so far:
# config/database.yml
development:
adapter: postgresql
encoding: unicode
database: myapp_development
pool: 10
username: myuser
password:
legacy_development:
adapter: mysql2
encoding: utf8
reconnect: false
database: myapp_legacy
pool: 10
username: myuser
password:
socket: /tmp/mysql.sock
# app/models/legacy.rb
class Legacy < ActiveRecord::Base
self.abstract_class = true
establish_connection "legacy_#{Rails.env}".to_sym
def self.import
self.find_each do |object|
imported_model = object.class.model.new object.attribute_map
object.report_failures unless imported_model.save
end
end
def self.import_all
Rails.application.eager_load!
self.subclasses.each {|subclass| subclass.import }
end
end
# app/models/legacy/chapter.rb
# a bunch of different subclasses like this
class Legacy::Chapter < Legacy
self.table_name = 'chapters'
def self.model
'Chapter'.constantize
end
def attribute_map
{
id: id,
name: name,
body: chapterBody
}
end
end
Then I have a rake task that runs Legacy.import_all. A lot of this was stolen from this post.
There's a few things wrong with this:
The main problem is, when i run Legacy.import_all it makes it through about half of the tables then I get an error like:
NoMethodError: undefined method 'import' for Legacy::SomeSubclass(Table doesn't exist):Class
I think this is because we just have too many connections in the pool. It seems though like it is looking for the SomeSubClass table_name from within the postgres database, but it should be looking on the mysql database.
This is probably because of the methods like:
def self.model
'Chapter'.constantize
end
in the above subclass. I am doing it like that instead of:
def self.model
Chapter
end
because I have a normal model (non-legacy) in my app also called Chapter and I was running into scoping issues there as well.
Anyways this is a huge mess and any thoughts about where I should dig would be greatly appreciated.
Thanks
Can you try prefixing subclass.import with ::
def self.import_all
Rails.application.eager_load!
self.subclasses.each {|subclass| ::subclass.import }
end
I have model to store database connection parameters (host, database name, username, password) and filling it by form. Before create or update I need to check if connection be good with entered parameters. I create validate :check_connection validator:
# don`t change primary connection
class Remote < ActiveRecord::Base; end
def check_connection
return if errors.any? || (!new_record? && password.empty?)
begin
Remote.establish_connection(
adapter: 'mysql2',
host: host,
username: username,
password: password,
database: database,
connect_timeout: 5,
reconnect: false
)
# maybe need to make some sql request? did try it
rescue Exception => e
errors.add :connection, 'Could not connect to database'
end
end
When I try enter accessible host (localhost), code like above works good. But if host like 192.168.1.1 (unaccessible) page freezing after submit form. I see attempts to connect every second and it did not stop after ESC (stop loading page) at browser (I see attempts at tcpdump on network interface). And attempts did not stop..
So, how can I validate connection to database? And if connection can not be established page must will not be load long time.
I did use gem 'mysql2' and bundle install 0.3.11 version. This version ignore connect_timeout and bug fixed at newer version. After I try 0.3.12b4 (gem 'mysql2', '~> 0.3.12b4') all works fine.
Variable connect_timeout is a global variable. Therefore, mysql2 maybe ignore it.
on mysql5.6:
mysql[(none)]> set connect_timeout = 123;
ERROR 1229 (HY000): Variable 'connect_timeout' is a GLOBAL variable and should be set with SET GLOBAL
I set timeout variables when initializing mysql2 but it's not reflected. A README of mysql2 says that you can set the *timeout options, but I think the README is outdated or broken.
on mysql2 0.3.14(gem):
client = Mysql2::Client.new(
host: 'localhost',
database: 'test',
username: 'root',
password: '',
connect_timeout: 3,
read_timeout: 3,
write_timeout: 3,
wait_timeout: 3);
client.query('show variables like "%timeout%"').map{|r| [r["Variable_name"], r["Value"]] }
=> [["connect_timeout", "10"],
["delayed_insert_timeout", "300"],
["innodb_lock_wait_timeout", "50"],
["innodb_rollback_on_timeout", "OFF"],
["interactive_timeout", "28800"],
["lock_wait_timeout", "31536000"],
["net_read_timeout", "30"], # Maybe older mysql has read_timeout?
["net_write_timeout", "60"], # Maybe older mysql has write_timeout?
["slave_net_timeout", "3600"],
["wait_timeout", "28800"]]
If you use ActiveRecord, you can set only wait_timeout variable by database.yml.
in database.yml:
development:
adapter: mysql2
encoding: utf8
charset: utf8
database: test
pool: 5
username: root
password:
host: localhost
connect_timeout: 3
read_timeout: 3
write_timeout: 3
wait_timeout: 3
A result of ActiveRecord 4.0.1:
> ActiveRecord::Base.connection.execute('show variables like "%timeout%"').to_a
=> [["connect_timeout", "10"],
["delayed_insert_timeout", "300"],
["innodb_flush_log_at_timeout", "1"],
["innodb_lock_wait_timeout", "50"],
["innodb_rollback_on_timeout", "OFF"],
["interactive_timeout", "28800"],
["lock_wait_timeout", "31536000"],
["net_read_timeout", "30"],
["net_write_timeout", "60"],
["rpl_stop_slave_timeout", "31536000"],
["slave_net_timeout", "3600"],
["wait_timeout", "3"]]
ActiveRecord set wait_timeout variable in abstract_mysql_adapter.rb.
see:
abstract_mysql_adapter.rb
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
mysql2_adapter.rb
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb
I am trying to connect multiple database in ROR application.My database.yml is look like this
in your database.yml file
development:
adapter: mysql
username: root
password:
database: example_development
private:
adapter: mysql
username: root
password:
database: example_private_development
It is possible to connect using establish_connection :private
My doubt is that how use rake db:create.I am not able get solution from google.
Please help me to clear it.
Try
rake db:create:all
And yes, it's possible to have multiple db connections in a Rails application.
This is what I did once, I have created two classes which inherit from ActiveRecord::Base and set the connections inside those classes.
Then I inherited all my models in one of those classes instead of direct ActiveRecord
Below is an example:
database.yml file
#app uses two database
#1 - test1
#2 - test2
test1:
adapter: mysql
encoding: utf8
database: test1
username: root
password: xxx
host: localhost
test2:
adapter: mysql
encoding: utf8
database: test2
username: root
password: xxx
host: localhost
Then I have two models for both test1 and test2 databases:
class Test1Base < ActiveRecord::Base
self.abstract_class = true
establish_connection("test1")
end
class Test2Base < ActiveRecord::Base
# No corresponding table in the DB.
self.abstract_class = true
establish_connection("test2")
end
Then I inherit my models according to database:
class School < Test1Base
#code
end
class Student < Test2Base
#code
end
Thanks for reply.
we can migrate a model for particular DB, for example
db:migrate RAILS_ENV="portal_development"'.
And more change for establishing connection with DB.check the corrected below
class Test1Base < ActiveRecord::Base
self.abstract_class = true
establish_connection :development
end
class Test2Base < ActiveRecord::Base
# No corresponding table in the DB.
self.abstract_class = true
establish_connection :portal_development
end
Thanks sameera for your valuable reply.
cheers
Shamith c
Possibly use active_delegate?
http://railslodge.com/plugins/595-active-delegate