db:schema:load vs db:migrate with capistrano - mysql

I have a rails app that I'm moving to another server and I figure I should use db:schema:load to create the mysql database because it's recommended. My problem is that I'm using capistrano to deploy and it seems to be defaulting to rake db:migrate instead. Is there a way to change this or is capistrano using db:migrate for a good reason?

Why to use db:schema:load
I find that my own migrations eventually do some shuffling of data (suppose I combine first_name and last_name columns into a full_name column, for instance). As soon as I do any of this, I start using ActiveRecord to sift through database records, and your models eventually make assumptions about certain columns. My "Person" table, for instance, was later given a "position" column by which people are sorted. Earlier migrations now fail to select data, because the "position" column doesn't exist yet.
How to change the default behavior in Capistrano
In conclusion, I believe deploy:cold should use db:schema:load instead of db:migrate. I solved this problem by changing the middle step which Capistrano performs on a cold deploy. For Capistrano v2.5.9, the default task in the library code looks like this.
namespace :deploy do
...
task :cold do
update
migrate # This step performs `rake db:migrate`.
start
end
...
end
I overrode the task in my deploy.rb as follows.
namespace :deploy do
task :cold do # Overriding the default deploy:cold
update
load_schema # My own step, replacing migrations.
start
end
task :load_schema, :roles => :app do
run "cd #{current_path}; rake db:schema:load"
end
end

Climbing up on the shoulders of Andres Jaan Tack, Adam Spiers, and Kamiel Wanrooij, I've built the following task to overwrite deploy:cold.
task :cold do
transaction do
update
setup_db #replacing migrate in original
start
end
end
task :setup_db, :roles => :app do
raise RuntimeError.new('db:setup aborted!') unless Capistrano::CLI.ui.ask("About to `rake db:setup`. Are you sure to wipe the entire database (anything other than 'yes' aborts):") == 'yes'
run "cd #{current_path}; bundle exec rake db:setup RAILS_ENV=#{rails_env}"
end
My enhancements here are...
wrap it in transaction do, so that Capistrano will do a proper rollback after aborting.
doing db:setup instead of db:schema:load, so that if the database doesn't already exist, it will be created before loading the schema.

That's a great answer from Andres Jaan Tack. I just wanted to add a few comments.
Firstly, here's an improved version of Andres' deploy:load_schema task which includes a warning, and more importantly uses bundle exec and RAILS_ENV to ensure that the environment is set up correctly:
namespace :deploy do
desc 'Load DB schema - CAUTION: rewrites database!'
task :load_schema, :roles => :app do
run "cd #{current_path}; bundle exec rake db:schema:load RAILS_ENV=#{rails_env}"
end
end
I have submitted a feature request to have deploy:load_schema implemented in Capistrano. In that request, I noted that the 'db:schema:load vs. db:migrate' debate has already been covered in the Capistrano discussion group, and there was some reluctance to switch the deploy:cold task to using db:schema:load over db:migrate, since if run unintentionally, the former nukes the entire database whereas the latter would probably complain and bail harmlessly. Nevertheless db:schema:load is technically the better approach, so if the risk of accidental data loss could be mitigated, it would be worth switching.

In Capistrano 3 / Rails 4, the default deploy syntax has changed. You can do this instead:
desc 'Deploy app for first time'
task :cold do
invoke 'deploy:starting'
invoke 'deploy:started'
invoke 'deploy:updating'
invoke 'bundler:install'
invoke 'deploy:db_setup' # This replaces deploy:migrations
invoke 'deploy:compile_assets'
invoke 'deploy:normalize_assets'
invoke 'deploy:publishing'
invoke 'deploy:published'
invoke 'deploy:finishing'
invoke 'deploy:finished'
end
desc 'Setup database'
task :db_setup do
on roles(:db) do
within release_path do
with rails_env: (fetch(:rails_env) || fetch(:stage)) do
execute :rake, 'db:setup' # This creates the database tables AND seeds
end
end
end
end
If you're cautious of invoking the standard deploy tasks manually in the :cold task (as they may change in upcoming version or if you have a custom deploy task), you can also simply call deploy:db_setup before running deploy.
To perform db:schema:load instead of db:setup, you can simply change the rake task, like so:
desc 'Load DB Schema'
task :db_schema_load do
...
execute :rake, 'db:schema:load'
...
end

Related

Migration error with devise

Hey I have a problem with my page here. I used a gem 'devise' to create a migration file for "users" I might have forgotten to rake db:migrate after that but I'm really not too sure what I did here to duplicate anything.
I ran the code rails g devise user I may have forgotten to db:migrate and then ran the code rails g devise:views
It's for a TeamTreeHouse project, and I'm sorry if I was too confusing with my question...anyway here is the error message.
== AddDeviseToUsers: migrating ===============================================
-- change_table(:users, :email)
rake aborted!
An error has occurred, this and all later migrations canceled:
SQLite3::SQLException: duplicate column name: email: ALTER TABLE "users" ADD "email" varchar(255) DEFAULT '' NOT NULL
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
It seems you already have an "users" table
Try to run: rake db:reset to run your migrations after drop and recreate your database.
UPDATE
This command will clear all data you have stored (unless is present in your seeds.rb), be careful to use this if you have important data in your database.
his command will clear all data you have stored (unless is present in your seeds.rb), be careful to use this if you have important data in your database.

spork/guard reload schema

I use spork and guard to speed up the test suite in my Rails 3.2 application. However the tests in guard behave strange after adding a new migration: they act as they don't know about the changes in schema, even though I run rake db:migrate before. They start to behave correctly when I stop guard and run rake spec. I suppose that it behaves this way, because spork/guard doesn't update schema before tests. Is it possible to configure spork to update schema on #prefork and guard to reload spork on schema changes?
You need to run rake db:test:prepare to apply migrations for testing environment.
Theoretically the following default Spork setup code should reload schema each time a test is run. Do you have it?
ActiveRecord::Schema.verbose = false
silence_stream STDOUT do
load Rails.root.join('db', 'schema.rb') # use db agnostic schema by default
load Rails.root.join('db', 'seeds.rb')
end

rspec tests are failing b/c application_controller is making db calls

I have some simple rspec tests that check if a particular page's title is 'xxxx'.
I added some before_filter's to my application_controller which makes db calls.
Should I be mocking these out somehow?
The tests are failing saying "mysql2::error table myapp_test.articles doesn't exist.
That sounds more like you need to run rake db:test:prepare or rake db:migrate db:test:prepare to me. Do you have something set up so your rspecs fail when they hit the database intentionally?

db:seed not loading models

I'm trying to seed my database with the standard db/seeds.rb method. This works fine on my development machine, but on my server, I get:
$ sudo rake db:seed RAILS_ENV=production --trace
** Invoke db:seed (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:seed
rake aborted!
uninitialized constant Permission
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2503:in `const_missing'
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.4/lib/active_support/dependencies.rb:92:in `const_missing'
/path/.../.../.../.../db/seeds.rb:4
/usr/local/lib/ruby/gems/1.8/gems/rails-2.3.4/lib/tasks/databases.rake:215:in `load'
/usr/local/lib/ruby/gems/1.8/gems/rails-2.3.4/lib/tasks/databases.rake:215
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
...
But when I check in the console, the model does exist:
$ script/console production
Loading production environment (Rails 2.3.4)
>> Permission
=> Permission(id: integer, ..., created_at: datetime, updated_at: datetime)
What am I forgetting?
In a comment to the blog posted linked to above and here again: http://www.builtfromsource.com/2011/02/09/getting-rake-dbseed-and-config-threadsafe-to-play-nice/
Bruce Adams mentions that one can call:
config.threadsafe! unless $rails_rake_task
to only turn on threadsafe when not running a rake task.
But as the issue is really that threadsafe turns off dependency_loading, you can simply add this line after config.threadsafe! to leave it enabled, but still load your environment as you expect.
config.dependency_loading = true if $rails_rake_task
It can be fixed by disabling threadsafe! in the environment configuration.
I just ran across a good approach to this problem in this article. I'll summarize here so people can (hopefully) find it quicker.
The idea is to turn off threadsafe in the production environment, first by editing config/environments/production.rb:
config.threadsafe! unless ENV['THREADSAFE'] == 'off'
You then set THREADSAFE=off when running rake tasks.

Rolling back a failed Rails migration

How do you roll back a failed rails migration? I would expect that rake db:rollback would undo the failed migration, but no, it rolls back the previous migration (the failed migration minus one). And rake db:migrate:down VERSION=myfailedmigration doesn't work either. I've ran into this a few times and it's very frustrating. Here's a simple test I made to duplicate the problem:
class SimpleTest < ActiveRecord::Migration
def self.up
add_column :assets, :test, :integer
# the following syntax error will cause the migration to fail
add_column :asset, :test2, :integer
end
def self.down
remove_column :assets, :test
remove_column :assets, :test2
end
end
result:
== SimpleTest: migrating =====================================================
-- add_column(:assets, :test, :integer)
-> 0.0932s
-- add_column(:asset, :error)
rake aborted!
An error has occurred, all later migrations canceled:
wrong number of arguments (2 for 3)
ok, lets roll it back:
$ rake db:rollback
== AddLevelsToRoles: reverting ===============================================
-- remove_column(:roles, :level)
-> 0.0778s
== AddLevelsToRoles: reverted (0.0779s) ======================================
huh? that was my last migration before SimpleTest, not the failed migration. (And oh, it would be nice if the migration output included the version number.)
So lets try running the down for the failed migration SimpleTest:
$ rake db:migrate:down VERSION=20090326173033
$
Nothing happens, and no output either. But maybe it ran the migration anyway? So lets fix the syntax error in the SimpleTest migration, and try to run it again.
$ rake db:migrate:up VERSION=20090326173033
== SimpleTest: migrating =====================================================
-- add_column(:assets, :test, :integer)
rake aborted!
Mysql::Error: Duplicate column name 'test': ALTER TABLE `assets` ADD `test` int(11)
Nope. Obviously the migrate:down didn't work. It's not failing, it's just not executing.
No way to get rid of that duplicate table other than manually going into the database and removing it, and then running the test. There's got to be a better way than that.
Unfortunately, you must manually clean up failed migrations for MySQL. MySQL does not support transactional database definition changes.
Rails 2.2 includes transactional migrations for PostgreSQL. Rails 2.3 includes transactional migrations for SQLite.
This doesn't really help you for your problem right now, but if you have a choice of database on future projects, I recommend using one with support for transactional DDL because it makes migrations much more pleasant.
Update - this is still true in 2017, on Rails 4.2.7 and MySQL 5.7, reported by Alejandro Babio in another answer here.
OK, folks, here's how you actually do it. I don't know what the above answers are talking about.
Figure out which part of the up migration worked. Comment those out.
Also comment out/remove the part of the migration that broke.
Run the migration again. Now it will complete the non-broken parts of the migration, skipping the parts that have already been done.
Uncomment the bits of the migration you commented out in step 1.
You can migrate down and back up again if you want to verify that you've got it right now.
To go to a specified version just use:
rake db:migrate VERSION=(the version you want to go to)
But if a migration fails part way, you'll have to clean it up first. One way would be:
edit the down method of the migration to just undo the part of the up that worked
migrate back to the prior state (where you started)
fix the migration (including undoing your changes to the down)
try again
I agree that you should use PostgreSQL when possible. However, when you are stuck with MySQL, you can avoid most of these problems by trying your migration on your test database first:
rake db:migrate RAILS_ENV=test
You can revert to the previous state and try again with
rake db:schema:load RAILS_ENV=test
At 2015 with Rails 4.2.1 and MySQL 5.7, a failed migration can't be fixed with standard rake actions that Rails provide, as it was at 2009.
MySql does not support rollback of DDL statments (at MySQL 5.7 Manual). And Rails can not do anything with that.
Also, we can check how Rails is doing the job: A migration is wrapped in a transaction depending on how connection adapter respond to :supports_ddl_transactions?. After a search of this action at rails source (v 4.2.1), I found that only Sqlite3 and PostgreSql supports transactions, and by default it is not supported.
Edit
Thus the current answer to the original question: A failed MySQL migration must be manually fixed.
The easy way to do this is to wrap all of your actions in a transaction:
class WhateverMigration < ActiveRecord::Migration
def self.up
ActiveRecord::Base.transaction do
...
end
end
def self.down
ActiveRecord::Base.transaction do
...
end
end
end
As Luke Francl noted, "MySql['s MyISAM tables don't] support transactions" -- which is why you might consider avoiding MySQL in general or at least MyISAM in particular.
If you're using MySQL's InnoDB, then the above will work just fine. Any errors in either up or down will back out.
BE AWARE some types of actions cannot be reverted via transactions. Generally, table changes (dropping a table, removing or adding columns, etc.) cannot be rolled back.
Run just the down migration from the console:
http://gilesbowkett.blogspot.com/2007/07/how-to-use-migrations-from-console.html (click through to his pastie)
I had a typo (in "add_column"):
def self.up
add_column :medias, :title, :text
add_colunm :medias, :enctype, :text
end
def self.down
remove_column :medias, :title
remove_column :medias, :enctype
end
and then your problem (cannot undo partly failed migration). after some failed googling i ran this:
def self.up
remove_column :medias, :title
add_column :medias, :title, :text
add_column :medias, :enctype, :text
end
def self.down
remove_column :medias, :title
remove_column :medias, :enctype
end
as you can see i just added the correction line by hand, and then removed it again, before i checked it in.
Alejandro Babio's answer above provides the best current answer.
One additional detail I want to add:
When the myfailedmigration migration fails, it is not considered as applied, and this can be verified by running rake db:migrate:status, which would show output similar to the following:
$ rake db:migrate:status
database: sample_app_dev
Status Migration ID Migration Name
--------------------------------------------------
up 20130206203115 Create users
...
...
down 20150501173156 Test migration
The residual effect of add_column :assets, :test, :integer being executed on the failed migration will have to be reversed at the database level with a alter table assets drop column test; query.