I'm trying to write a rake task for loading data into my database. My plan is do something like
system "mysql -u foo -pbar database < backup.sql"
but I need to access the config/database.yml data for getting the user, pass and database info. The trick is I don't want to "parse" this file but to access this info in the same way tasks like rake db:reset do.
How can I do that?
This will work.
task :demo_using_db_config => :environment do
db_config = Rails.application.config.database_configuration[Rails.env]
system "mysql -u#{db_config['username']} -p#{db_config['password']} #{db_config['database']} < backup.sql"
end
Since this is environment specific, we want the task to depend on the :environment task.
Alternatively, if you are using ActiveRecord, you can get to this information like below:
abcs = ActiveRecord::Base.configurations
puts abcs[Rails.env]["username"]
puts abcs[Rails.env]["password"]
This works in rake tasks and elsewhere.
In Rails 3, from the console, you can type the following to see various database login credentials you defined in database.yml
config = Rails.application.config.database_configuration
In your rake file, you could specify something like this:
task :mysqlimport_table
db_config = Rails.application.config.database_configuration[Rails.env]
sh "mysqlimport -u#{db_config['username']} -p#{db_config['password']} #{db_config['database']} --default-character-set=utf8 --local <path_to_your_file>"
end
task :mysqlrun_sql_script
db_config = Rails.application.config.database_configuration[Rails.env]
sh "mysql -u#{db_config['username']} -p#{db_config['password']} -D#{db_config['database']} < <path_to_your_sql_file>"
end
Like this:
require 'yaml'
conf = YAML.load_file("path/to/database.yml")
Related
I know the command in Rails 4+, it's
rails db < ./path/to/db_dump.sql
but I don't know how to do it with Rails 2.3. I know it's old, but it's all I have to work with.
I cannot use the MySQL CLI. I have to use a Rake task because the application is dockerized, so my MySQL container doesn't "know" about my rails_app_development database.
Does anyone know the Rails 2.3 command for this, if there is one?
Things I've tried:
1. This one i feel like has gotten me the closest
namespace :db do
task :import_from_source => :environment do
Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8
ActiveRecord::Base.connection.execute(IO.read("./file.sql"))
end
end
So i basically just made a rake task. Then in docker I can run:
$ docker-compose run app rake db:import_from_source
And it appears that everything worked. In fact, I see this in my logs:
app_1 | -- Dump completed on 2020-07-24 6:20:02
app_1 |
app_1 | SQL (0.7ms) SET SQL_AUTO_IS_NULL=0
However when I refresh the page, I still get a "mysql table dosen't exist" error
2. This doesn't work and I'm not sure why
First I run:
docker-compose run --rm app rake db:create
and that works, but when I go into my db container using docker-compose run db bash and run
mysql -u root -p rails_db_development < /db_dump/db_dump.sql
I get this error:
ERROR 1049 (42000): Unknown database 'rails_db_development'
Even though I created it in the app container.
Hi I have a ruby on rails app hosted on heroku and it is using mysql as database.
Now I have to take backup of the database to my local machine. But I am getting issues while taking backup.
For this I installed taps gem and I am using following commands for it
heroku pg:pull mysql2://username#hostname.cleardb.com/heroku_database local_database --app my_app
but it is giving error as !Your app has no databases.
Can any one guide me how to pull mysql database from heroku to local machine.
EDIT
I have used following syntax for the command
heroku pg:pull <REMOTE_SOURCE_DATABASE> <LOCAL_TARGET_DATABASE>
and for getting REMOTE_SOURCE_DATABASE I have used following command
heroku config:get DATABASE_URL --app my_app
I refer this link1 and link2 for more detailed heroku documentation.
The pg:pull command only works with Postgres databases in your Heroku app. But, you are using a third-party MySQL provider. Your database is hosted on the ClearDB servers and it's available to anyone with the right credentials, including both your app server on Heroku and your dev machine.
Even though there aren't special commands to pull the database, you don't need any - plain mysqldump should do.
mysqldump -h hostname.cleardb.com -u username heroku_database | mysql local_database
Running $heroku config | grep ^DATABASE will give you something like this:
DATABASE_URL: mysql2://username:password#host/dbname?reconnect=true`
From there you can construct your db dump command:
mysqldump -h host -p -u username dbname | mysql local_database
This will prompt you for the password which you received from the previous command. If you wanted to create a script that would automatically include the password from the heroku command you could do something like this:
mysqldump -u username --password=`heroku config | grep ^DATABASE | sed 's/.*[a-z0-9][a-z0-9]*:\([a-z][a-z0-9]*\).*/\1/'` -h host dbname | mysql cedric
In this way you can have a script that will import the database without requiring any user input but also does not expose the password to your database.
(IMPORTANT DISCLAIMER: You MUST have your database.yml configured correctly in order for this to work. I am not responsible for any data you lose as a result of running the below script.)
For Ruby on Rails users ... you could consider writing a Rake task like these db:clone tasks below.
I find myself using this script constantly to clone down from production to development. It's way easier than remembering the mysqldump syntax, much less all of the usernames and passwords involved ...
To clone from production to development:
rake db:clone:production
To clone from staging to development:
rake db:clone:staging
To clone from production to staging:
rake db:clone:production_to_staging
And here's the code enjoy (and be careful in setting up your database.yml):
namespace :db do
namespace :clone do
class << self
%w(development test staging production).each do |env|
define_method("#{env}_db") do
Rails.configuration.database_configuration[env]
end
end
end
def clone_db(from_db, to_db)
start_time = Time.now
puts "Cloning Remote DB...."
system("mysqldump -h#{from_db['host']} -u#{from_db['username']} -p#{from_db['password']} #{from_db['database']} | mysql #{to_db['database']} -u#{to_db['username']} -p#{to_db['password']}")
puts "Import Successful"
end_time = Time.now
puts "===================="
puts "Job Completed: #{end_time - start_time} Seconds"
end
task :staging => :environment do
clone_db(staging_db, development_db)
end
task :production => :environment do
clone_db(production_db, development_db)
end
task :production_to_staging => :environment do
clone_db(production_db, staging_db) if Rails.env.staging?
end
end
end
I know there are many mysql-proxy questions on SO, however I have read through many of them and none seem to solve my problem. I am simply trying to get mysql-proxy up and running, with the eventual purpose of rewriting some queries that go through the proxy. I am using ubuntu 14.04, I have mysql-proxy version 0.8.1, and mysql version 5.5.37. To start mysql-proxy I run the following line on the command line
sudo mysql-proxy --defaults-file=mysql-proxy.cnf
where the file mysql-proxy.cnf looks like the following:
[mysql-proxy]
log-file= /var/log/mysql/proxy-error.log
log-level= debug
admin-lua-script= /usr/lib/mysql-proxy/lua/admin.lua
proxy-lua-script= /path/to/lua/script/example.lua
admin-username = myusername
admin-password = mypassword
proxy-skip-profiling = true
proxy-address = localhost:4040
proxy-backend-addresses = localhost:3306
plugins = proxy,admin
My example.lua script is very simple, and meant only to verify that the mysql-proxy query is being altered. example.lua is pasted below
-- first_example.lua
function read_query(packet)
if string.byte(packet) == proxy.COM_QUERY then
print("Hello world! Seen the query: " .. string.sub(packet, 2))
end
end
Since I don't run this with the --daemon flag, when I run that line above in the command line it just loops indefinitely, which is expected.
Finally, in separate terminal session, I run the following on the command line and enter my password in order to connect with the proxy
mysql -u myusername -p -h localhost -P 4040
I then select a database to use, and run a simple SELECT query on one of the tables. Based on multiple articles/tutorials I've read on mysql-proxy, my first console session, the one that ran mysql-proxy, should print out some data based on the example.lua file. However this does not happen, in fact nothing happens.
I'm not sure if the following bit of information makes any difference, but in my "my.cnf" mysql configuration file, I have these couple of lines
bind-address = 255.255.255.255
#bind-address = 127.0.0.1
where I have replaced my actual ip address with 255.255.255.255 because I do not want to display my ip address publicly.
Please, I have been trying to figure this out for several days, and no amount of new lua scripts, or changing the host:port parameters in the mysql-proxy.cnf file have solved anything. I
Working on an app that wil be used by admin only to create multiple instances of another app.
In other words, app A will be used to create clones of app B on server and create all the configurations needed to run that cloned app on a subdomain new_clone.domain.com.
Managed to clone the app, created the apache config file, unicorn server settings file too. Managed to run rake db:create; db:migrate but I did this with the root user of mysql.
At the point where I clone the app, I generate a database.yml file for the new cloned app that has at this moment username and password set as root, but I woould like to have a different user for each cloned app.
The app A has a model called Subdomain, in the subdomains_controller.rb at the create action I do all the things, clone, generate config files, running rake tasks, etc... and also in this create action I need to create a new Mysql user with privileges for a specific database.
What I have tried so far, can't say I did much, I tried to run the mysql command within controller create action:
def create
if #subdomain.save
....
system "mysql -u root -p root; create database new_database;..."
....
else
....
end
end
but this puts my create action on hold until I type in the password, and even if I'll find a way to go over it I am not sure the rest of mysql commands will work. Probably there is a better way to add a Mysql user with one command line without going into mysql console.
Thank you.
As far as I understood, your create method runs under Rails app with root mysql user.
Then you can simply execute mysql commands via AR adapter:
# Create DB
ActiveRecord::Base.connection.execute("CREATE DATABASE IF NOT EXISTS \
`#{ActiveRecord::Base.sanitize(db_name)}` CHARACTER SET = `utf8` \
COLLATE = `utf8_general_ci`")
# Create User
ActiveRecord::Base.connection.execute("REPLACE INTO mysql.user \
(Host, User, Password) VALUES(\"localhost\", \
\"#{ActiveRecord::Base.sanitize(db_user)}\", \
PASSWORD(\"#{ActiveRecord::Base.sanitize(db_password)}\"))")
# Grant access
ActiveRecord::Base.connection.execute("GRANT ALL PRIVILEGES ON \
`#{ActiveRecord::Base.sanitize(db_name)}`.* TO \
`#{ActiveRecord::Base.sanitize(db_user)}`")
# Apply
ActiveRecord::Base.connection.execute("FLUSH PRIVILEGES")
Thanks to Yuriy's answer I got it working in a shorter way:
def create_mysql_user
#db_name = "#{#subdomain.name}_domain_production"
#db_user = "#{#subdomain.name}_user"
#db_password = "#{#subdomain.name}domain_password"
ActiveRecord::Base.connection.execute("GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER,
INDEX ON `#{#db_name}`.* TO #{ActiveRecord::Base.sanitize(#db_user)}#localhost
IDENTIFIED BY #{ActiveRecord::Base.sanitize(#db_password)};")
# Apply
ActiveRecord::Base.connection.execute("FLUSH PRIVILEGES")
end
I would like to create a ruby script that I can run mysql commands on a remote server through a ssh tunnel.
Right now I have a manual process to do this:
Create a tunnel -> ssh -L 3307:127.0.0.1:3306
run ruby script.
Close tunnel.
I would love to be able to automate this so I can just run the script.
example:
require 'rubygems'
require 'net/ssh/gateway'
require 'mysql'
#make the ssh connection -> I don't think I am doing this right.
Net::SSH.start('server','user') do |session|
session.forward.local(3307,'127.0.0.1', 3306)<br>
mysql = Mysql.connect("127.0.0.1","root","","",3307)
dbs = mysql.list_dbs<br>
dbs.each do |db|<br>
puts db <br>
end
session.loop(0){true}<br>
end
An update - 2010-11-10:
I'm really close with this code:
require 'rubygems'
require 'mysql'
require 'net/ssh/gateway'
gateway = Net::SSH::Gateway.new("host","user",{:verbose => :debug})
port = gateway.open("127.0.0.1",3306,3307)
# mysql = Mysql.connect("127.0.0.1","user","password","mysql",3307)
# puts "here"
# mysql.close
sleep(10)
gateway.close(port)
When its sleeping, I am able to open a terminal window and connect to mysql on the remote host. This verifies the tunnel is created and working.
The problem now is when I uncomment the 3 lines, it just hangs.
I was able to get this to work without a fork using the mysql2 gem
require 'rubygems'
require 'mysql2'
require 'net/ssh/gateway'
gateway = Net::SSH::Gateway.new(
'remotehost.com',
'username'
)
port = gateway.open('127.0.0.1', 3306, 3307)
client = Mysql2::Client.new(
host: "127.0.0.1",
username: 'dbuser',
password: 'dbpass',
database: 'dbname',
port: port
)
results = client.query("SELECT * FROM projects")
results.each do |row|
p row
end
client.close
This might be one possible solution:
require 'rubygems'
require 'mysql'
require 'net/ssh/gateway'
gateway = Net::SSH::Gateway.new("server","user")
port = gateway.open("127.0.0.1",3306,3307)
child = fork do
mysql = Mysql.connect("127.0.0.1","user","password","mysql",port)
sql = "select sleep(5)"
mysql.query(sql)
mysql.close
exit
end
puts "child: #{child}"
Process.wait
gateway.close(port)
Maybe there is a better way, but this works for what I was trying to do.
I've been trying out the gateway code above, one main difference being I have to use ssh keys for passwordless access, but also found the code handing on the Mysql.connect statement. However, when I replaced
mysql = Mysql.connect("127.0.0.1",...)
with
mysql = Mysql.connect("localhost",...)
it worked fine.
My final code looks like this:
require 'rubygems'
require 'mysql'
require 'net/ssh/gateway'
gateway = Net::SSH::Gateway.new('host',
'user',
:keys => ['myprivatekey.pem'],
:verbose => :debug)
port = gateway.open("127.0.0.1",3306,3307)
mysql = Mysql.connect("localhost","dbuser","dbpassword","dbname",3307)
puts "here"
mysql.close
gateway.close(port)
gateway.shutdown!
Usually, when a tunnel is up binding a local port to the remote application port, you just connect to the local port as if it were the remote one. Remember that MySQL has access policies based on the source location of the connection, so you might want to keep that in mind. In my opinion, there's no session.forward.local nessessary.
Of course, you still don't speak the MySQL connection protocol so this might not be what you want. It might be easier to drop whatever queries to run into a file, then run mysql -u"user" -p"password"
You can also try this nice ruby gem: https://github.com/progrium/localtunnel