I have this project and I need to integrate a bunch of databases into my Rails project.
This is the database configuration that I have to work with.
$ nano config/database.yml
production:
adapter: mysql2
reconnect: true
pool: 5
username: user_xyz
password: 123456
database: database1
host: localhost
database_2:
adapter: mysql2
reconnect: false
pool: 5
username: user_xyz
password: 123456
database: database2
host: 192.168.2.100
database_3:
adapter: mysql2
reconnect: false
pool: 5
username: user_xyz
password: 123456
database: database3
host: 192.168.2.101
database_4:
adapter: mysql2
reconnect: false
database: database4
pool: 5
username: user_xyz
password: 123456
host: 192.168.2.102
I need to update the schema inside db/schema.rb but unfortunately it's only producing the schema for production (database1).
$ RAILS_ENV=production bundle exec rake db:schema:dump
I can't just run this as it complains:
$ bundle exec rake db:schema:dump
rake aborted!
database configuration does not specify adapter
What do I need to do to get all of these databases into db/schema.rb?
Austio's answer is a good one. Here is some extra detail on how to implement a solution. Create a new rake file, e.g., lib/tasks/schema_dump.rake, and add the following contents:
namespace :db do
namespace :schema do
desc 'dumps the schema of database_1 to db/schema_db1.rb'
task :dump_db1 => :environment do
ActiveRecord::Base.establish_connection 'database_4'
File.open(Rails.root.join('db', 'schema_db1.rb'), 'w') do |file|
ActiveRecord::SchemaDumper.dump ActiveRecord::Base.connection, file
end
end
end
end
Now when you run rake db:schema:dump it'll dump the schema of your primary database to db/schema.rb (like traditional Rails). But when you run rake db:schema:dump_db1, it'll dump the schema of your database_1 block into db/schema_db1.rb.
You can create similar tasks for your other database blocks and have a schema file for each database.
You will need to establish a connection to each database that you would like to schema dump on. Here is example for database_4
ActiveRecord::Base.establish_connection 'database_4'
Then when you run schema dump, it will dump the current database you have a connection with. I'm not sure if the rake task has arguments to copy it to a different file name, so you may have to rename the file.
Related
I'm new to Ruby and ActiveRecord, so this may be not a good question.
I'm currently working on a project with Ruby on Rails. And when I take a look into projects query summaries, this looks like called in each transaction by ActiveRecord.
SET NAMES utf8mb4 COLLATE utf8mb4_general_ci, ##SESSION.sql_mode = CONCAT(CONCAT(##sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'), ##SESSION.sql_auto_is_null = 0, ##SESSION.wait_timeout = 2147483
I'm wondering if I can put this query into some config file and make it global, and disable this config query.
This is like a 5th most called query, so disabling this will improve performance.
You can try this by adding the following to your config/database.yml file:
default: &default
adapter: mysql2
encoding: utf8mb4
collation: utf8mb4_general_ci
variables:
sql_mode: STRICT_ALL_TABLES,NO_AUTO_VALUE_ON_ZERO
sql_auto_is_null: 0
wait_timeout: 2147483
development:
<<: *default
database: myapp_development
test:
<<: *default
database: myapp_test
production:
<<: *default
database: myapp_production
username: myapp
Configuring the database connection in the database.yml file will prevent ActiveRecord from setting the configuration in each connection. But it will still set the configuration in the connection pool. So if you have a lot of connections, you will still see the query in the query log. You can prevent this by setting the configuration in the database server instead.
Example for MySQL in the my.cnf file:
[mysqld]
sql_mode=STRICT_ALL_TABLES,NO_AUTO_VALUE_ON_ZERO
sql_auto_is_null=0
wait_timeout=2147483
I have used pgloader for converting the mysql database to postgres database. The mysql database contains the valid data for both created_at and updated_at fields of each table, while the postgres database contains inconsistent data for created_at and updated_at fields. I wish to import data without having any kind of manipulation to the mysql database.
I have tried changing the code in controller and the views to get the desired output, but I think it should be feasible to change the data stored in the database instead of using conditions and adding code to controller and views.
I used
rails db:create
rails db:migrate
to create the postgres database. My database.yml looks like:
default: &default
adapter: postgresql
# pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
# timeout: 5000
# port: 5432
username: postgres_username
password: postgres_password
host: 'localhost'
development:
<<: *default
database: tech_login_development_mysql
encoding: utf8
pool: 50
test:
<<: *default
encoding: utf8
database: tech_login_development_mysql
pool: 5
production:
<<: *default
encoding: utf8
database: tech_login_development_mysql
pool: 10
So now the tech_login_development_mysql database is created and all the migrations are executed. After creating the postgres database structure I used pgloader to load the imported mysql data. I have created a file named as database_data.load and its code is given below:
LOAD DATABASE
FROM mysql://mysql_username:mysql_password#localhost/tech_login_development_mysql
INTO postgresql://postgres_username:postgres_password#localhost/tech_login_development_mysql
ALTER SCHEMA 'tech_login_development_mysql' RENAME TO 'public';
After creating the above file I ran:
pgloader database_data.load
command to load the data in tech_login_development_mysql database
There is a Log table is my project whose created_at and updated_at fields in the newly created postgres database is not same as mysql database. When i ran:
Log.find(40124).created_at
Actual Result is:
Tue, 03 Apr 2018 08:10:04 IST +05:30
Expected Result is:
Tue, 03 Apr 2018 13:40:04 IST +05:30
So there is a timezone issue while migrating data from mysql to postgres, but I can't seem to find any solution.
You have to add
CAST type datetime to timestamp drop default drop not null using zero-dates-to-null;
at the end of your database_data.load
I am using ruby-2.2.4, Rails 4.2.5 and MySQL 5.7.16 with gem mysql2 in my Ruby on Rails application. I have created database with name 123_4 and set database name in /config/database.yml.
Why I am getting error ActiveRecord::NoDatabaseError: Unknown database '1234' when trying rake db:migrate?
If I try to run rake db:create database with name 1234 will be created.
If I use 123_abc4 for database name everything is fine.
my database.yml content:
production:
adapter: mysql2
database: 123_4
host: localhost
username: user
password: "pass"
encoding: utf8
Identifiers may begin with a digit but unless quoted may not consist solely of digits.
MySQL Schema Object Name
So you can use 123_abc4 because it contains letters.
If it only includes number, you will need to quote it: '123_4'
I have a new rails 4.2 app.
In order to perform a migration from an old database to the new one, I added a rake task that defines some OldXX classes connecting to the old database, converts data and saves it to new database. The task works from localhost in development against old database on the server.
convert.rake
class OldActiveRecord < ActiveRecord::Base
self.abstract_class = true
establish_connection "old_#{Rails.env}".to_sym
end
class OldUser < OldActiveRecord
...
end
...
database.yml which is not present in / linked to shared on production.
default: &default
adapter: mysql2
encoding: utf8
pool: 5
production:
<<: *default
host: localhost
database: "<%= ENV['DATABASE_NAME'] %>"
username: "<%= ENV['DATABASE_USERNAME'] %>"
password: "<%= ENV['DATABASE_PASSWORD'] %>"
old_production: &old_prod
<<: *default
host: my.old.dbhost
database: "<%= ENV['OLD_DATABASE_NAME'] %>"
username: "<%= ENV['OLD_DATABASE_USERNAME'] %>"
password: "<%= ENV['OLD_DATABASE_PASSWORD'] %>"
old_development:
<<: *old_prod
old_test:
<<: *old_prod
and the env vars are set up in .env which is in shared on production.
Problem
When I deploy with capistrano I'm getting the error
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as me#my.old.dbhost: rake exit status: 1
rake stderr: rake aborted!
ActiveRecord::AdapterNotSpecified: 'old_production' database is not configured. Available: []
/var/www/vhosts/domain.org/www.domain.org/shared/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/connection_adapters/connection_specification.rb:250:in `resolve_symbol_connection'
/var/www/vhosts/domain.org/www.domain.org/shared/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/connection_adapters/connection_specification.rb:211:in `resolve_connection'
/var/www/vhosts/domain.org/www.domain.org/shared/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/connection_adapters/connection_specification.rb:139:in `resolve'
/var/www/vhosts/domain.org/www.domain.org/shared/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/connection_adapters/connection_specification.rb:169:in `spec'
/var/www/vhosts/domain.org/www.domain.org/shared/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/connection_handling.rb:50:in `establish_connection'
/
right after do bundle exec rake assets:precompile.
Adding a p Rails.configuration.database_configuration before establish_connection shows that all fields for old_production are set accordingly, including adapter. Connection from new host to old host for mysql is also possible, tested with the same credentials on command line.
Why am I getting this error and how to solve it?
Update
When I add a p configurations in /var/www/vhosts/domain.org/www.domain.org/shared/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/connection_adapters/connection_specification.rb:245 at the top of #resolve_symbol_connection it says
{}
I had to use
ActiveRecord::Base.configurations = Rails.application.config.database_configuration
at the top of convert.rake in order to make it work, but actually I still don't know why this needs to be done.
Im making an application with a pre-existing database, I set up the database.yml to use the database.
database.yml
development:
adapter: mysql2
encoding: utf8
# database: ttlem_demo_development
database: ttle
pool: 5
username: root
password:
socket: /tmp/mysql.sock
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
adapter: mysql2
encoding: utf8
database: ttlem_demo_test
pool: 5
username: root
password:
socket: /tmp/mysql.sock
production:
adapter: mysql2
encoding: utf8
database: ttlem_demo_production
pool: 5
username: root
password:
socket: /tmp/mysql.sock
I only want one table out of the database it is called account views, I try to generate a scaffold for this with all the correct fields but it tells me i need to migrate (when i render it in the browser), if i migrate i wont be able to use the existing data, is this correct? How can i make a model that will use the existing data and fields?
Thank you for all your help :)
Two things to try:...
#1 Run your scaffold without a migration so it doesn't think you're missing one.
rails g scaffold yourmodelname fieldone:fieldtype ... etc --no-migration
#2 If that doesn't work you can go the long way round but dumping and reloading with a valid schema version number
Add this db to yml to your gemfile:
gem 'yaml_db', github: 'jetthoughts/yaml_db', ref: 'fb4b6bd7e12de3cffa93e0a298a1e5253d7e92ba'
It works for either rails 3 or rails 4.
Do a schema dump of your current database so you'll get a schema.rb with a valid version number.
bundle exec rake db:schema:dump
Now that you have a valid schema dump your data.
bundle exec rake db:data:dump
Drop your database (you can do it manually using the mysql commands if you prefer or run rake db:drop)
Now recreate it with your schema file.
bundle exec rake db:schema:load
Now add back your data
bundle exec rake db:data:load
Start your server and assuming your correctly matched all your data fields in your model (so the proper strong parameters are set from your scaffold) you should be good.