Check mysql connection to remote host - mysql

I have model to store database connection parameters (host, database name, username, password) and filling it by form. Before create or update I need to check if connection be good with entered parameters. I create validate :check_connection validator:
# don`t change primary connection
class Remote < ActiveRecord::Base; end
def check_connection
return if errors.any? || (!new_record? && password.empty?)
begin
Remote.establish_connection(
adapter: 'mysql2',
host: host,
username: username,
password: password,
database: database,
connect_timeout: 5,
reconnect: false
)
# maybe need to make some sql request? did try it
rescue Exception => e
errors.add :connection, 'Could not connect to database'
end
end
When I try enter accessible host (localhost), code like above works good. But if host like 192.168.1.1 (unaccessible) page freezing after submit form. I see attempts to connect every second and it did not stop after ESC (stop loading page) at browser (I see attempts at tcpdump on network interface). And attempts did not stop..
So, how can I validate connection to database? And if connection can not be established page must will not be load long time.

I did use gem 'mysql2' and bundle install 0.3.11 version. This version ignore connect_timeout and bug fixed at newer version. After I try 0.3.12b4 (gem 'mysql2', '~> 0.3.12b4') all works fine.

Variable connect_timeout is a global variable. Therefore, mysql2 maybe ignore it.
on mysql5.6:
mysql[(none)]> set connect_timeout = 123;
ERROR 1229 (HY000): Variable 'connect_timeout' is a GLOBAL variable and should be set with SET GLOBAL
I set timeout variables when initializing mysql2 but it's not reflected. A README of mysql2 says that you can set the *timeout options, but I think the README is outdated or broken.
on mysql2 0.3.14(gem):
client = Mysql2::Client.new(
host: 'localhost',
database: 'test',
username: 'root',
password: '',
connect_timeout: 3,
read_timeout: 3,
write_timeout: 3,
wait_timeout: 3);
client.query('show variables like "%timeout%"').map{|r| [r["Variable_name"], r["Value"]] }
=> [["connect_timeout", "10"],
["delayed_insert_timeout", "300"],
["innodb_lock_wait_timeout", "50"],
["innodb_rollback_on_timeout", "OFF"],
["interactive_timeout", "28800"],
["lock_wait_timeout", "31536000"],
["net_read_timeout", "30"], # Maybe older mysql has read_timeout?
["net_write_timeout", "60"], # Maybe older mysql has write_timeout?
["slave_net_timeout", "3600"],
["wait_timeout", "28800"]]
If you use ActiveRecord, you can set only wait_timeout variable by database.yml.
in database.yml:
development:
adapter: mysql2
encoding: utf8
charset: utf8
database: test
pool: 5
username: root
password:
host: localhost
connect_timeout: 3
read_timeout: 3
write_timeout: 3
wait_timeout: 3
A result of ActiveRecord 4.0.1:
> ActiveRecord::Base.connection.execute('show variables like "%timeout%"').to_a
=> [["connect_timeout", "10"],
["delayed_insert_timeout", "300"],
["innodb_flush_log_at_timeout", "1"],
["innodb_lock_wait_timeout", "50"],
["innodb_rollback_on_timeout", "OFF"],
["interactive_timeout", "28800"],
["lock_wait_timeout", "31536000"],
["net_read_timeout", "30"],
["net_write_timeout", "60"],
["rpl_stop_slave_timeout", "31536000"],
["slave_net_timeout", "3600"],
["wait_timeout", "3"]]
ActiveRecord set wait_timeout variable in abstract_mysql_adapter.rb.
see:
abstract_mysql_adapter.rb
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb
mysql2_adapter.rb
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb

Related

Why can't my rake task connect to my MySQL instance during a test run?

TL;DR
Why does this mysqldump rake task successfully connect to MySQL and finish executing when I run it directly in the terminal (in both development environment and with RAILS_ENV=test prefix), but not when I run it inside of an RSpec test?
Long Version
I have a Rails app with the following new rake task:
EXCLUDED_TABLES = %w(
google_oauth_tokens
intuit_oauth_tokens
oauth_access_grants
oauth_access_tokens
oauth_applications
oauth_openid_requests
qbwc_sessions
tokens
)
namespace :db do
desc "Export all tables in the DB except for ones we specify into a compressed file that can be pulled down onto a developer's local machine"
task :export_allowed_tables => :environment do
directory = File.join(Rails.root, 'db', 'dump')
FileUtils.mkdir directory unless File.exists?(directory)
db = YAML::load( File.open( File.join(Rails.root, 'config', 'database.yml') ) )[ Rails.env ]
filename_to_use = File.join( directory, "#{Rails.env}_allowlisted_#{DateTime.now.to_s}.sql" )
username_flag = "-u #{ERB.new(db['username']).result}"
password_flag = "-p#{ERB.new(db['password']).result}"
excluded_tables_flag = EXCLUDED_TABLES.map{|table| "--ignore-table=#{ERB.new(db['database']).result}.#{table}"}.join(" ")
database_param = "#{ERB.new(db['database']).result}"
command = "mysqldump -v #{username_flag} #{password_flag} #{excluded_tables_flag} #{database_param} > #{filename} | gzip > #{filename}.gz"
sh command
end
I also have the following test to ensure my rake task works as expected:
require 'spec_helper'
describe 'rake db:export_allowed_tables' do
after do
files = Dir.glob("db/dump/*allowlisted*.sql")
files.each do |file|
expect(File.delete(file)).to eq 1
end
end
it 'exports the expected data' do
test_full_name = 'Foo Bar Baz Test User'
test_user = FactoryBot.create(:user, full_name: test_full_name)
expect(Dir.glob("db/dump/*.sql")).to be_empty
MyApp::Application.load_tasks
Rake::Task['db:export_allowed_tables'].invoke
expect(Dir.glob("db/dump/*.sql")).not_to be_empty
end
end
The task works fine when I run rake db:export_allowed_tables, and it even works fine when I prefix the command with RAILS_ENV=test. My SQL file shows up as expected, with 7000+ lines of code ending in Dump completed on 2020-05-22 8:44:29, and in my terminal the script starts off with:
mysqldump: [Warning] Using a password on the command line interface can be insecure.
-- Connecting to localhost...
-- Retrieving table structure for table access_control_analytics_reports_claims...
-- Sending SELECT query...
-- Retrieving rows...
...
However, when I run my spec file, the script hangs on the following line:
mysqldump: [Warning] Using a password on the command line interface can be insecure.
-- Connecting to localhost...
And while the .sql file is created in the expected folder, it's empty when I open it (which is to be expected if a connection to localhost failed).
I'm also able to open my Rails console in the terminal (again, both with and without the RAILS_ENV=test prefix) and execute the following without any problems connecting to localhost:
MyApp::Application.load_tasks
Rake::Task['db:export_allowed_tables'].invoke
I am able to create that instance of User, so I know the test environment database is up and running. And I'm able to run the rake task even when MySQL is running, so it doesn't seem to be an issue of the MySQL port being in-use when I run the task. What could be causing my test failure?
Here's my database.yml file, for reference:
default: &default
adapter: <%= ENV["MYAPP_DB_ADAPTER"] || "mysql2_annotator" %>
username: <%= ENV["DATABASE_USERNAME"] || ENV["MYSQL_USER"] || ENV['MYAPP_USER'] || [*FILTERED*] %>
password: <%= ENV["DATABASE_PASSWORD"] || ENV["MYSQL_PASSWORD"] || ENV['MYAPP_PASSWORD'] || [*FILTERED*] %>
database: <%= ENV['MYAPP_DB'] || ENV["DATABASE"] || ENV["MYSQL_DATABASE"] || "myapp_dev#{ENV['MYAPP_ENV_NUMBER']}" %>
host: <%= ENV["MYSQL_HOST"] || ENV['DATABASE_HOST'] || 'localhost' %>
port: <%= ENV['MYAPP_DB_PORT'] || ENV['DATABASE_PORT'] || ENV["MYSQL_PORT"] || 3306 %>
encoding: utf8mb4
collation: utf8mb4_unicode_ci
strict: true
reconnect: true
variables:
sql_mode: "STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION"
pool: 20
timeout: 20000
checkout_timeout: 20
development:
<<: *default
test:
<<: *default
database: <%= ENV['MYAPP_TEST_DB'] || "myapp_test" %><%= ENV['TEST_ENV_NUMBER'] %>

sequelize dialectoptions , UTC and dates, for mysql

I am still learning how to use sequelize.
I am using sequelize 5 along with node 10 and mysql2 .
I am setting my connection like so
const sequelize = new Sequelize({
database: dbs,
username: usr,
host: hst,
password: pwd,
port: prt,
dialect: 'mysql',
dialectOptions: {
useUTC: true,
dateFirst: 1
},
define:{
timestamps:false,
paranoid:false,
freezeTableName: true
}
});
and I am getting the following warnings
Ignoring invalid configuration option passed to Connection: useUTC.
This is currently a warning, but in future versions of MySQL2, an
error will be thrown if you pass an invalid configuration option to a
Connection
Ignoring invalid configuration option passed to Connection: dateFirst.
This is currently a warning, but in future versions of MySQL2, an
error will be thrown if you pass an invalid configuration option to a
Connection
If I remove the useUTC: true and dateFirst: 1 from the dialectOptions, the errors obviously go away.
What am I missing and how can I set those options for mysql

RubyOnRails multiple database connections errors

I have a rails application (ruby 2.0.0, Rails 4.0.1) that connects to several MySQL database.
The connection to the local database always work properly, this is my configuration:
production:
adapter: mysql
encoding: utf8
database: xxx
username: yyy
password: zzz
host: x.x.x.x
port: 3306
reconnect: true
But connections to remote databases often return an error, such as connecting to my radius external database returns the following error:
Mysql :: Error:: SELECT 1 AS one FROM WHERE `` radacct` radacct`.`username` =? LIMIT 1
Updating the page several times, the connection is restored and I can see the data properly. This is my configuration:
radius:
port: 3306
adapter: mysql
encoding: utf8
database: xxx
username: yyy
password: zzz
host: x.x.x.x
reconnect: true
connect_timeout: 60
read_timeout: 90
write_timeout: 90
checkout_timeout: 10
reaping_frequency: 30
dead_connection_timeout: 30
I tried to change the configuration of the timers in different ways, but the problem persists.
to connect to the radius server I created the following model common to all:
class RadActiveRecord <ActiveRecord :: Base
self.abstract_class = true
establish_connection "radius"
end
for example for the table radacct use the following model:
class RadAcctDetail <RadActiveRecord
self.table_name = "radacct"
end
the error in question is with any queries, such as:
def index
#rad_acct_details = RadAcctDetail.all.paginate = (
: order => "radacctid DESC"
: page => params [: page] || 1
: per_page => params [: per_page] || 25)
end
Does anyone have any suggestions?

Accessing another database wtih SQL queries

I am attempting to access another database in my rails app, as well as query that database with SQL statements and return JSON.
The main database and secondary are on the same server running in MySQL. When testing to see if this would simply work as:
The datagbase.yml file:
development:
adapter: mysql2
encoding: utf8
database: onlineportal
username: root
password:
host: 127.0.0.1
port: 3306
socket:
android_mobile_developement:
adapter: mysql2
encoding: utf8
database: androidchatterdatabase
username: root
password:
host: 127.0.0.1
port: 3306
socket:
but then building out the method in the controller:
class RequestsController < ApplicationController
def getActivatedFriends
#results = User.find_by_sql("SELECT
a.id
, a.username
, a.phoneNumber
FROM androidchatterdatabase.users a
WHERE phoneNumber in (8754444444) and
removed = 0 and
is_user = 1;")
respond_to do |format|
format.html
format.json { render json: { friends_match: #results }}
end
end
and calling the route:
localhost:3000/getActivatedFriends/
from the routes:
match '/getActivatedFriends/', to: 'requests#getActivatedFriends', via: 'get'
yields the error:
Missing template requests/getActivatedFriends, application/getActivatedFriends with {:locale=>[:en], :formats=>[:html], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :jbuilder]}.
While sending the request, send it as local host:3000/getActivatedFriends.json.That way controller will respond to json request. By default it responds to html request.
You can access another database by specifying the database name before the table, such as:
SELECT * FROM onlineportal.table
- or -
SELECT * FROM androidchatterdatabase.table
Is this what you're looking to do? If they're on the same server this syntax would work.

Are database read_timeout and write_timeout unique to ActiveRecord or native to mysql?

ActiveRecord allows you to configure database read_timeout and write_timeout values, for example:
production:
adapter: mysql
encoding: utf8
database: mydb
pool: 5
username: myuser
password: mypass
host: myhost
write_timeout: 2
read_timeout: 10
Are read_timeout and write_timeout unique to ActiveRecord, or are those part of the mysql spec? I know that mysql has the concept of innodb_lock_wait_timeout, but I don't believe that is the same as query timeouts.
Thanks!
I have never used mysql gem. But I think you can see mysql variables which are used by ActiveRecord like this.
> ActiveRecord::Base.connection.execute('show variables like "%timeout%"').to_a
=> [["connect_timeout", "10"],
["delayed_insert_timeout", "300"],
["innodb_flush_log_at_timeout", "1"],
["innodb_lock_wait_timeout", "50"],
["innodb_rollback_on_timeout", "OFF"],
["interactive_timeout", "28800"],
["lock_wait_timeout", "31536000"],
["net_read_timeout", "30"],
["net_write_timeout", "60"],
["rpl_stop_slave_timeout", "31536000"],
["slave_net_timeout", "3600"],
["wait_timeout", "2147483"]]