I am using rails and the mysql2 adapter. I want to change all primary ids and foreign keys to be 64 bit integers instead of the default 32 bit as they are right now for my production database.
Is this possible on the fly or do I have to drop the database, change the structure and import the data again?
If there is a way to do it without dropping the database, even if it's a hack, it would be great to know.
Rails 5.1 already added a bigint type for migrations, you can do this:
change_column :users, :id, :bigint
Source:
http://www.mccartie.com/2016/12/05/rails-5.1.html
While ActiveRecord does not support this, you are able to do it using execute
class UpdateUserIdLimit < ActiveRecord::Migration
def up
# PostgreSQL
execute('ALTER TABLE users ALTER COLUMN id SET DATA TYPE BIGINT')
# MySQL
execute('ALTER TABLE users MODIFY COLUMN id BIGINT(8) NOT NULL AUTO_INCREMENT')
end
def down
raise ActiveRecord::IrreversibleMigration
end
end
For new tables you should be able to simply do
def change
create_table :users, id: false do |t|
t.int :id, limit: 8, primary_key: true
t.string :first_name
t.string :last_name
end
end
Also starting with Rails 5.1 primary keys will be BIGINT by default.
Related
With activerecord (< 5.0, >= 3.2) I was able to use the activerecord-mysql-unsigned gem to create an UNSIGNED INT in my MySQL database, but there's been no updates to that gem and I can't find any documentation regarding native support for that in Rails 5.
Is there an options hash or something that can be called in the add_column method which allows this to be done?
There is an option for unsigned integer, bigint, decimal, and float in the schema adapter for MySQL as of Rails 5.1.x
Something like this in your migration will work in Rails 5.1.4
def up
create_table :unsigned_columns do |t|
t.integer "positive", :unsigned => true
t.timestamps
end
end
If are using rails > 5.1 and you want to have all your primary keys being generated with unsigned integer as default type, all that you have to do is adapt your generator in your config/application.rb , adding the following code:
config.generators do |g|
# ... other configurations
g.orm :active_record, primary_key_type: :unsigned_integer
end
With that a call to rails generate model foo bar:string will create the id as unsigned.
I wrote a migration with the following:
class CreateTableSomeTable < ActiveRecord::Migration[5.1]
def change
create_table :some_tables do |t|
t.references :user, foreign_key: true
t.references :author, references: :user, foreign_key: true
t.text :summary
end
end
end
It is a basic migration that is creating a database table. However: when I run rails db:migrate a very odd error message aborts the migration:
Mysql2::Error: Table 'my_database.some_tables' doesn't exist: SHOW FULL FIELDS FROM 'some_tables'
It is as if the error is saying it can't create the table because the table does exist, which doesn't make sense.
Things I have looked at and tried:
reviewed the database.yml which seems fine. Nothing has changed, and I have recently run other migrations just fine (though no migrations that created database tables)
ran bundle to ensure all gems were installed
deleted the schema.rb file, recreated the database with data from another copy, and I ran rake db:schema:dump to recreate the schema.rb file. I attempted to run the migration again and still got the same error.
I am using rails 5.1.1 as well as mysql2 0.4.6
Any tips on how I can get the migration to run?
I got a similar error when trying to create a new model that has a reference to an existing model that was created before migrating to Rails 5.1.
Although the error message was not very clear about that, in my case it turned out that the problem was data type mismatch between the primary key of the old model and the foreign key of the new model (MySQL does not permit that). It was so because since Rails 5.1 the default data type of all the primary and foreign keys is bigint, but for the old model the primary key type was still integer.
I solved this by converting all the primary and foreign keys of the current models to bigint, so I can use the Rails new defaults and forget about it.
A workaround could also be specifying integer type for the new foreign keys so that they match the primary keys type of the old models. Something like the following:
class CreateUserImages < ActiveRecord::Migration[5.1]
def change
create_table :user_images do |t|
t.references :user, type: :integer, foreign_key: true
t.string :url
end
end
end
The big issue with the ActiveRecord migration 5.1 is that now the id are expected to be BIGINT instead of INT, so when you adding a column referring another table created before rails 5.1 it expect the column type to be BIGINT but instead is just an INT, hence the error.
The best solution is just modify your migration and change the type of the column to int.
class CreateTableSomeTable < ActiveRecord::Migration[5.1]
def change
create_table :some_tables do |t|
t.references :user, foreign_key: true, type: :int
t.references :author, references: :user, foreign_key: true
t.text :summary
end
end
that should work.
I figured out a work around, but it is still very puzzling to me.
The error message in the log file was not exactly pointing to the issue. For some reason, it might be rails 5.1.1 or it might be mysql2 0.4.6, but it doesn't like using references within the create_table block for some reason. Very odd because it has worked for me in the past.
So I changed the migration from this:
class CreateTableSomeTable < ActiveRecord::Migration[5.1]
def change
create_table :some_tables do |t|
t.references :user, foreign_key: true
t.references :author, references: :user, foreign_key: true
t.text :summary
end
end
end
To this:
class CreateTableSomeTable < ActiveRecord::Migration[5.1]
def change
create_table :some_tables do |t|
t.integer :user_id
t.integer :author_id
t.text :summary
end
end
end
And it worked.
It is very odd because references works just fine with sqlite3 (I tested this by generating a dummy app, ran a scaffold command with a references column, and ran rails db:migrate and it all worked).
This drove me nuts, I think I was seeing a different reason for this than what others suggested. In my case it happened because my migration file names didn't exactly match the migration class therein. For example, I had a migration file named 20171205232654_bonus.rb but inside the class was declared as class CreateBonus < ActiveRecord::Migration[5.1]. Once I changed the file name to 20171205232654_create_bonus.rb everything worked.
This might have something to do with the fact that I've been creating migrations only, not full scaffolds, and maybe I did something wrong. I really don't know how I wound up with that mismatch.
I have a migration for a model with a binary field meant to store a file that can be bigger than 10Mb:
class CreateNewModel < ActiveRecord::Migration
def change
create_table :new_model do |t|
...
t.binary :data, limit: 16777216
...
end
end
end
With the limit information the migration can create a longblob object in a MySQL or MariaDB database, as seen in How do you get Rails to use the LONGBLOB column in mysql?.
The migration seems to work fine on a MariaDB database: data has longblob type. However loading directly from the schema gives data a blob type instead of longblob, this means the rake db:setup command is no longer usable as the schema doesn't reflect the database I want.
This seems pretty evident when one looks at the db/schema.rb file:
create_table "new_model", force: :cascade do |t|
...
t.binary "data"
...
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
end
There is no limit info and as such loading the schema can only lead to blobs and not longblobs.
Why doesn't the limit information get written in the schema?
I don't want to change the schema manually as it would mean have to redo the change at each migration (as they regenerate the schema file). What other solutions do I have, is there a way to force the limit field from the migration to the schema?
I've tried using the shorthands described in https://github.com/rails/rails/pull/21688 but it doesn't seem to exist in Rails 4.2.6.
I am using Rails 3.2.6 and Mysql 6.0.9 (but I have exactly the same error on MySQL 5.2.25)
When I create new database (rake db:create) and then when I try to load the schema (rake schema:load) I get this error:
Mysql2::Error: Specified key was too long; max key length is 767 bytes: CREATE UNIQUE INDEX `unique_schema_migrations` ON `schema_migrations` (`version`)
After hours and hours of research I found these solutions:
1. Change MySQL variable innodb_large_prefix to true (or ON)
This didn't work. I tried it on my Linux server, my Mac and even on Windows - it just doesn't work.
2. Monkeypatch ActiveRecord::SchemaMigration.create_table
I do not need the version column to be 255 long (when it is UTF-8, then it takes 4*255 = 1020 bytes and exceeds the MySQL limit of 767 byte for keys). I do not need it to be UTF-8 either, but all other tables in the DB are UTF-8 and I have set utf8_czech_ci to be the default collation.
The method that actually creates the schema_migrations table looks like this:
def self.create_table
unless connection.table_exists?(table_name)
connection.create_table(table_name, :id => false) do |t|
t.column :version, :string, :null => false
end
connection.add_index table_name, :version, :unique => true, :name => index_name
end
end
You can read the whole file on Github rails/rails
So I tried to add :limit => 100 to the t.column statement, but I did not succeed with this solution either. The problem is that I cannot make this patch load when the originial is already in place. In other words - my patch loads before ActiveRecord::SchemaMigration so it is overwritten.
When I put this in config/initializers/patches/schema_migration.rb:
require 'active_record/scoping/default'
require 'active_record/scoping/named'
require 'active_record/base'
module ActiveRecord
class SchemaMigration < ActiveRecord::Base
def self.create_table
unless connection.table_exists?(table_name)
connection.create_table(table_name, :id => false) do |t|
t.column :version, :string, :null => false, :limit => 100
end
connection.add_index table_name, :version, :unique => true, :name => index_name
end
end
end
end
It is successfully loaded, but the it is overwritten when the original ActiveRecord::SchemaMigration is loaded.
I tried to mess up with ActiveSupport.on_load(:active_record) but that doesn't seem to work either.
Is there a way to load this file after the originial ActiveRecord::SchemaMigration is in place and make this patch work?
Do you have any suggestions? I can clarify any part of this question, if it makes no sense to you. Just ask me. I've been stuck with this for too long.
767 key should work. Make sure you use utf8 encoding, and not utf16.
I had same problem, and my mistake was that I accidently created utf16 database
I suggest you to drop your database and recreate a new one with the following instructions :
mysql -u root -p -e "CREATE DATABASE {DB_NAME} DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;"
I have the same problem with a column named version for varchar of length 2000
class AddVersionToUsers < ActiveRecord::Migration
def change
add_column :users, :version, :string, limit:2000
add_index :users, :version
end
end
I was using this latin 1 1 character 1 byte, but now I want to use utf8mb4 1 character 4 bytes.
Configuring your databse like this you can get index until 3072 bytes:
docker run -p 3309:3306 --name test-mariadb -e MYSQL_ROOT_PASSWORD=Cal1mero. -d mariadb:10.2 --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --innodb-large-prefix=1 --innodb-file-format=barracuda --innodb-file-per-table=1 --innodb-strict-mode=1 --innodb-default-row-format=dynamic
this is enough for latin_1, (will be 2000 bytes), but for utf8mb4 it will be 8000 bytes. In this keys you have some options
Add a column named hash_version and implement the index on that column.
Consistent String#hash based only on the string's content
Make the string shorter, it should work , but depernds on your needs
or use fulltext in your migrations, like this:
class AddVersionToUsers < ActiveRecord::Migration
def change
add_column :users, :version, :string, limit:2000
add_index :users, :version, type: :fulltext
end
end
references:
https://mensfeld.pl/2016/06/ruby-on-rails-mysql2error-incorrect-string-value-and-specified-key-was-too-long/
https://codex.wordpress.org/Converting_Database_Character_Sets
https://dev.mysql.com/doc/refman/8.0/en/innodb-restrictions.html
https://docs.oracle.com/cd/E17952_01/mysql-5.7-en/innodb-restrictions.html
https://dba.stackexchange.com/questions/35821/possible-index-on-a-varchar-field-in-mysql
We have some imported data from a 3rd party which provides non-integer unique ids.
In Rails 2.2.2 we were able to use :foreign_key on our has_many relationships with a non-integer column and it worked.
But we are now upgrading to Rails 2.3.8 and it seems to force the foreign_key to an integer. Has anyone found a way to get this working?
According to this answer, the procedure has to be completed using execute:
create_table :employees, {:id => false} do |t|
t.string :emp_id
t.string :first_name
t.string :last_name
end
execute "ALTER TABLE employees ADD PRIMARY KEY (emp_id);"
And as Sean McCleary mentioned, your ActiveRecord model should set the primary key using set_primary_key:
class Employee < ActiveRecord::Base
set_primary_key :emp_id
...
end