I am using dashing.io and want to grab some data from a mysql database using rufus scheduler to push them.
I have this code for testing the connection closing in one .rb job file:
require 'rubygems'
require 'mysql2'
SCHEDULER.every '12h', :first_in => 0 do |job|
begin
db = Mysql2::Client.new(:host => "localhost", :username => "root", :password => "rainer-zufall", :port => 2971, :database => "test" )
open_connections = db.query("show status like 'Connections';")
open_connections.each do |row|
puts "connections: #{row}"
end
rescue
ensure
db.close
end
end
The database connection do not close. The connection counter still increases and soon there will be an error stating max_connections reached from the mysql-server.
Can anybody help?
Related
I need to create a rake task that gets the table rows from a MySQL database, parse the data, and insert into an Oracle database. The databases are on two different hosts.
My current attempt:
namespace :import_from_mysql do
class MySQLConnection < ActiveRecord::Base
self.abstract_class = true
establish_connection({
:adapter => 'mysql',
:host => 'xxx.xxx.com',
:database => 'sample_database',
:username => 'username',
:password => 'password'
})
end
class MySQLTable < MySQLConnection
self.table_name = "users"
self.primary_key = "id"
self.inheritance_column = "something_unique_here"
end
desc "Parse data before inserting to oracle database"
task :insert_to_oracle => :environment do |t|
puts "Rake task has begun...\n\n"
puts "Parsing data from MYSQL...\n\n"
MySQLTable.establish_connection
puts "Rake task has completed!"
end
end
But the MySQLTable.establish_connection establishes a connection to my local database which is sqlite even though I'm trying to connect to a mysql_adapter.
When I tried to establish a connection using the command below, I was able to connect to a MySQL adapter but I don't know how I can access my tables after the connection was established:
ActiveRecord::Base.establish_connection({:adapter => "mysql", :database => "sample_database", :host => "xxx.xxx.com", :username => "username", :password => "password" })
Any idea on why it keeps on connecting to sqlite? And after successfully establishing a connection to mysql, how do I select table rows after the MySQLTable.establish_connection statement?
With the connection generated using ActiveRecord::Base you can execute SQL statements against whatever database you connect to. Like so:
connection = ActiveRecord::Base.establish_connection(
:adapter => "mysql",
:host => "faraway",
:username => "myuser",
:password => "mypass",
:database => "somedatabase"
)
connection.execute('SELECT * FROM users')
Once established, the connection can also be referenced from ActiveRecord::Base class.
ActiveRecord::Base.establish_connection(...)
ActiveRecord::Base.connection.execute(...)
You can use mysql2 gem (https://github.com/brianmario/mysql2) inside your rake task.
client = Mysql2::Client.new(:host => "localhost", :username => "username", :database => "sample_database", :password => "password")
users = client.query("SELECT * FROM users")
Thought it might be helpful for someone else. The following worked for me!
connection = ActiveRecord::Base.establish_connection(
:adapter => "mysql",
:host => "faraway",
:username => "myuser",
:password => "mypass",
:database => "somedatabase"
)
#connection = ActiveRecord::Base.connection
result = #connection.exec_query('SELECT * FROM users')
result.each do |row|
puts row
end
Here, users is an already existing table in the "somedatabase".
although I am a noob with connections and "database back office" my trivial approach works like a charm, and there is no need with parameters in rake task that is for updating "domain tables" that contain only defining data like legal or help texts:
pre_db = ActiveRecord::Base.establish_connection('preproduction').connection
prod_db = ActiveRecord::Base.establish_connection('online').connection
and in database.yml I defined the databases. ("preproduction" is a step between dev and prod in my case and a real environment like "development". where 'online' is almost the same as 'production' (I made a different entry only for security reasons - not to destroy production if database.yml gets uploaded)
I want to add a second read-only database to my application based on a variable aside the SQLite file I'm developing with.
So, there should be three databases.
1. Local SQLite file
2. Production Read-only MySQL database
3. Test Read-only MySQL database
This test database has new columns added and will be pushed as production database soon.
From this SO-Q, I do not understand what the solution is.
I already use establish_connection.
establish_connection(
:adapter => 'mysql2',
:database => "db1",
:username => "username",
:password => "p*ssw*rd",
:host => "ho.st.com"
)
This db1 is the production db without the extra table columns.
db2 is the test db with the extra table columns.
establish_connection(
:adapter => 'mysql2',
:database => "db2",
:username => "username",
:password => "p*ssw*rd",
:host => "ho.st.com"
)
I thought this could be a solution:
In the models of the tables from the MySQL databases:
config = 'test'
case config
when 'test'
establish_connection(
:adapter => 'mysql2',
:database => "db1",
:username => "username",
:password => "p*ssw*rd",
:host => "ho.st.com"
)
else
establish_connection(
:adapter => 'mysql2',
:database => "db2",
:username => "username",
:password => "p*ssw*rd",
:host => "ho.st.com"
)
end
But now I have to edit all 'affected' models when I change the status. Is there a way to variabilise (is that even a word?) this so I only have to edit this config once?
I've tried via Application_controller.rb, but this didn't work.
# Application_controller.rb
def get_config_status
return 'test'
end
# model.rb
config = get_config_status() #=> undefined method `get_config_status' for Regio(Table doesn't exist):Class
# or
config = Application.get_config_status #=> uninitialized constant Regio::Application
Or should I consider totally something else?
To be honest, I don't like putting usernames and passwords in Models. Is this a possible thread for hackers?
Summary
What I'm trying to accomplish is:
Set one variable so my application uses db2 instead of db1.
The octopus gem might be helpful.
https://github.com/tchandy/octopus
You can select the database at query level,
User.where(name: 'Sam').using(:db2)
Or you can select the database per controller action,
class ApplicationController < ActionController::Base
around_filter :select_shard
def select_shard(&block)
Octopus.using(:db1, &block)
end
end
Check out the readme on Github for more examples.
I'm dealing with this a few days, and cannot connect to a simple mysql database on localhost.
require "mysql"
#db_host = "localhost"
#db_user = "myrubyapp"
#db_pass = "1234"
#db_name = "myrubyapp"
mysql = Mysql.new(:host => #db_host, :username => #db_user, :password => #db_pass, :database => #db_name)
The output I got is an error: can't convert Hash into String (TypeError) where Mysql.new is.
The second one, I tried to change the gem to mysql2
require "mysql2"
#db_host = "localhost"
#db_user = "myrubyapp"
#db_pass = "1234"
#db_name = "myrubyapp"
mysql = Mysql2.new(:host => #db_host, :username => #db_user, :password => #db_pass, :database => #db_name)
The output is an error too, but is different from the first: undefined method "new" for Mysql2:Module (NoMethodError).
Guys I'm sorry that I have to ask this kind of questions, but I'm really really confused, I have an experience of programming more than 3 years in JavaSE and EE, I am ashamed cause I can't deal with that. Please point me into right direction and don't judge me harshly. I am new to Ruby.
Try
mysql = Mysql2::Client.new(:host => #db_host, :username => #db_user, :password => #db_pass, :database => #db_name)
correct syntax is:
client = Mysql2::Client.new(:host => "localhost", :username => "root")
see mysql2 on github for more examples
I recomment you to take a look at Sequel as raw mysql2 lib provide a very limited functionality and Sequel can do a lot.
Never used the gem mysql, but have you tried to remove the hash :host => etc and pass directly a list of parameters?
Something like con = Mysql.new db_host, db_user, db_pass, db_name
http://zetcode.com/db/mysqlrubytutorial/ for a tutorial
I have a Rails 3.2 app which use PostgreSQL to store all the information.
But in one Rake task I need to make a connection with the MySQL server. I tried to do this:
ActiveRecord::Base.establish_connection(
:adapter => "mysql2",
:database => "foo",
:user => "root",
:password => "",
)
But it just replace my default PostgreSQL connection with this temporary MySQL.
How to make the additional connection for the instance?
I found a very simple solution: to the the vanila mysql2 gem (https://github.com/brianmario/mysql2)
Now my code looks like:
client = Mysql2::Client.new(:host => "localhost", :username => "root", :database => "foobar", :password => "")
users = client.query("SELECT * FROM users")
After that I have an array of results.
Don't establish it on ActiveRecord::Base.
establish_connection connects to a database from a class, as you've discovered, so when you do it on AR:Base, every subclass of that (to whit, the entire database) has the connection established on it, replacing the current one.
Basically, you create a class for each of the tables you want to connect to, and call the establish connection method in those. If you want to do it in several tables, then create a module with it in and include it.
class MyCustomClass < ActiveRecord::Base
establish_connection(
:adapter => "mysql2",
:database => "foo",
:user => "root",
:password => "",
)
end
MyCustomClass.find(1)
My Rails 3.1 application connects to 2 databases, one is the default, the other is an Amazon RDS MYSQL instance.
The current database.yml contains two production database connections. The models that need to pull from the second database simply use
establish_connection "production_on_amazon"
Unfortunately Heroku overwrites your database.yml, and only seems to inlcude one database connection. Does anyone know how I can add or configure my second?
Running "heroku config" I can see there are 2 DB's listed but cant seem to configure to connect to both. Perhaps somehow set my default to the SHARED_DATABASE_URL db on Heroku and set the alternate to the DATABASE_URL which points to Amazon...
Working off of the previous responses, but incorporating some Rails 3 advantages with the configuration and simplifying the parsing...
# config/application.rb
module MyApp
class Application < Rails::Application
... other configs
config.secondary_database_url = ENV['SECONDARY_DB_URL']
end
end
We may want to override this in development / test
# config/environments/development.rb
module MyApp
class Application < Rails::Application
... other configs
config.secondary_database_url = 'SOME_CONNECTION_STRING'
end
end
Now to setup the class we'll have our models inherit from...
# lib/active_record/secondary.rb
module ActiveRecord
class Secondary < ActiveRecord::Base
self.abstract_class = true
# prior to AR 3.2.1
url = URI.parse( MyApp::Application.config.secondary_database_url )
establish_connection(
:adapter => 'mysql',
:host => url.host,
:username => url.userinfo.split(':')[0],
:password => url.userinfo.split(':')[1],
:database => url.path[1..-1],
:port => url.port || 3306
)
# as of AR 3.2.1
establish_connection(MyApp::Application.config.secondary_database_url)
end
class SecondaryMigration < ActiveRecord::Migration
def connection
ActiveRecord::Secondary.connection
end
end
end
Heroku will always connect your app to the production DB that they create for you. If you want to make an additional connection you'll need to do this in your code manually, and create a ENV var that the code can use as a connection string.
Anything in the production segment of database.yml is binned by Heroku and replaced.
Regarding Neil's answer, here is a way to do it. Not an out-of-box solution, but might give you an idea...
/lib/active_record_extensions.rb
module ActiveRecordExtensions
class Shard < ActiveRecord::Base
#need to switch to the shard database connection from heroku config
primary_database_url = ENV['PRIMARY_DATABASE_URL']
if(!primary_database_url.nil?)
parsed_connection_string = primary_database_url.split("://")
adapter = parsed_connection_string[0]
parsed_connection_string = parsed_connection_string[1].split(":")
username = parsed_connection_string[0]
parsed_connection_string = parsed_connection_string[1].split("#")
password = parsed_connection_string[0]
parsed_connection_string = parsed_connection_string[1].split("/")
host = parsed_connection_string[0]
database = parsed_connection_string[1]
establish_connection(
:adapter => adapter,
:host => host,
:username => username,
:password => password,
:database => database,
:port => 3306,
:pool => 5,
:timeout => 5000
)
else
self.establish_connection "shard_#{Rails.env}"
end
end
class ShardMigration < ActiveRecord::Migration
def connection
ActiveRecord::Shard.connection
end
end
end
So your model should just extend ActiveRecord::Shard instead of Base