Why does activerecord not populate an auto-incrementing column in the item returned from a create? - mysql

Why does rails not populate an auto-incrementing column in the item returned from a create? Is there a better way to do this?
In rails, when you do a = Foo.create then a.id is populated
But if you have a field that was created via
def up
execute "ALTER TABLE my_table ADD COLUMN my_auto_incrementing_column INTEGER AUTO_INCREMENT not null UNIQUE KEY;"
end
Then that field does not appear when you use create. You have to use a reload also.
a = Foo.create
a.id # not nil
a.my_auto_incrementing_column # nil
a.reload
a.my_auto_incrementing_column # is now populated
Version information:
$ ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-darwin14.5.0]
$ bundle exec rails -v
Rails 3.2.12
$ mysql --version
mysql Ver 14.14 Distrib 5.6.26, for osx10.10 (x86_64) using EditLine wrapper
Some background:
This code is being applied to a large existing in-production rails codebase that requires that all id fields be UUIDs. The auto_increment column is not a primary key, because it was added after we had discovered that a new external integration partner could not handle using our existing long unique identifiers (UUIDs).
We are working hard to update our version of ruby but we don't want to wait for that as a solution to this problem. Also, after reading changelogs in activerecord, I still don't have proof that any future version of ruby/rails will contain a bugfix for this problem.
The code which I want to improve:
class Foo < ActiveRecord::Base
has_one :object_containing_auto_incrementing_column
def my_method
if self.object_containing_auto_incrementing_column.nil?
self.object_containing_auto_incrementing_column = ObjectContainingAutoIncrementingColumn.create(owner: self)
self.object_containing_auto_incrementing_column.reload
end
self.object_containing_auto_incrementing_column.my_auto_incrementing_column
end
end

After looking at the source code it does not appear that ActiveRecord tries to populate auto-incrementing columns. It only assigns the value that is returned by the INSERT statement to the #id attribute and nothing else.
# ActiveRecord:: Persistence::ClassMethods
def create
# ...
self.id ||= new_id if self.class.primary_key
# ...
end
If you want to populate my_auto_incrementing_column without hitting the DB twice, I think there is no way around patching ActiveRecord itself.
Have a look at how the insert method is implemented:
# Returns the last auto-generated ID from the affected table.
#
# +id_value+ will be returned unless the value is nil, in
# which case the database will attempt to calculate the last inserted
# id and return that value.
#
# If the next id was calculated in advance (as in Oracle), it should be
# passed in as +id_value+.
def insert(arel, name = nil, pk = nil, id_value = nil, sequence_name = nil, binds = [])
sql, binds = sql_for_insert(to_sql(arel, binds), pk, id_value, sequence_name, binds)
value = exec_insert(sql, name, binds)
id_value || last_inserted_id(value)
end
There might not be any trivial way to change the current API to populate your field in question.

Apparently it is still not possible to get in create() a field (not id) generated in autoincrement at database level by SEQUENCE.
I worked around the problem
in my case: PostgreSQL AUTOINCREMENT by SEQUENCE
with an after_create callback
Migration
class CreateFoo < ActiveRecord::Migration[7.0]
def change
create_table :foo do |t|
t.integer :autoinc_field
end
execute "CREATE SEQUENCE table_name_seq OWNED BY table_name.autoinc_field INCREMENT BY 1 START WITH 100000"
execute "ALTER TABLE table_name ALTER COLUMN autoinc_field SET DEFAULT nextval('table_name_seq');"
end
end
Model
class Foo < ApplicationRecord
after_create :reload
end
Result
>> Foo.create!.autoinc_field
=> 100000

Related

How to check if the current user from devise has a certain attribute set

For this website I am using devise to create user profiles. Some users have special permission attributes. In this case db_admin_status is a Boolean attribute. If the user has this set to True, then I want certain nav-links to appear on the page.
This is what the user model looks like
Table name: users
#
# id :bigint not null, primary key
# db_admin_status :boolean
# email :string default(""), not null
# encrypted_password :string default(""), not null
# name :string
# remember_created_at :datetime
# reset_password_sent_at :datetime
# reset_password_token :string
# user_name :string
# created_at :datetime not null
# updated_at :datetime not null
For this rails project I am using application.html.erb and Bootstrap. So in my collapse-nav bar I have embedded in other if statements (that do work) this:
<%if current_user.db_admin_status == 'true'%>
<!--- links appear on nav bar-->
<%end%>
However even when my current user im signed in as has the attribute set to true, the links still don't show up
Step 1.)
You don't need to use any comparisons in the conditional statement as a boolean variable will just result to true/false by itself.
Simply doing
<% if current_user.db_admin_status %>
// your code
<% end
should suffice.
Step 2.)
You could run into trouble when the value of db_admin_status is nill (has not been set yet). You can fix this by defaulting the field to false in your database schema:
Run the following command to create a rails migration to update the db_admin_status field to default to false.
rails g migration DefaultDbAdminStatusToFalseOnUser
This should create a file db/migrate/20130201121110_default_db_admin_status_to_false_on_user.rb
class DefaultDbAdminStatusToFalseOnUser < ActiveRecord::Migration
def change
// your changes go here
end
end
Modify this file to update your field on the user table.
change_column :users, :db_admin_status, :boolean, :default => false
And then finally migrate your changes by running:
rake db:migrate
You want to compare your attribute as boolean and not string.
current_user.db_admin_status == true
Although i would create a helper method which would check if user is eligible to show link and call that method from your view.
def should_show_some_link?
current_user.db_admin_status? #which will return true or false
end
Also your column db_admin_status is not set to default, is that what you want? When the new user creates the value will be nil and not false.

Is it possible to bypass sql_color in Rails active record logs?

I have an issue where a binary representaiton of an IPv6 address will cause the sql_color in active record logs to generate a very long error message.
The query does work and does return expected results.
I think this is because it the binary IPv6 looks like:
"\xFE\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\f"
And the sql_color method is interpreting that as control codes.
If I set:
Rails.applicaiton.config.colorize_logging = false
It still does it even though there is no longer any color being shown.
Ideally I would like to know the best way to bypass sql_color.
Right now I have just overridden the method and put it in a begin/rescue block.
How to reproduce:
rails new binary_bug -d mysql
cd binary_bug
rails db:create
rails g model Bug name:text first_ip:binary second_ip:binary
Update the migration to
class CreateBugs < ActiveRecord::Migration[5.2]
def change
create_table :bugs do |t|
t.text :name
t.binary :first_ip, limit: 16
t.binary :second_ip, limit: 16
t.timestamps
end
end
end
rails db:migrate
Bug.create(name: 'test1', first_ip: IPAddr.new('fe80::c').hton, second_ip: IPAddr.new('fe80::c').hton.to_s )
Bug.create(name: 'test2', first_ip: IPAddr.new('2001:db8:1234::').hton, second_ip: IPAddr.new('2001:db8:1234:ffff:ffff:ffff:ffff:ffff').hton.to_s )
# Try to search the DB.
bugs = Bug.where("first_ip > ?", IPAddr.new('2001:db8:1234::12').hton)
This will give a very long error that starts with:
Could not log "sql.active_record" event. ArgumentError: invalid byte sequence in UTF-8
The first file it points towards is:
gems/activerecord-5.2.2/lib/active_record/log_subscriber.rb:71:in `sql_color'"
Which is a private method and looks like this:
def sql_color(sql)
case sql
when /\A\s*rollback/mi
RED
when /select .*for update/mi, /\A\s*lock/mi
WHITE
when /\A\s*select/i
BLUE
when /\A\s*insert/i
GREEN
when /\A\s*update/i
YELLOW
when /\A\s*delete/i
RED
when /transaction\s*\Z/i
CYAN
else
MAGENTA
end
end
If I replace that with just MAGENTA or wrap it in a begin/rescue block and restart spring it will work fine.

How I can drop an index if exist using django migrations.RunSQL and MySQL

I'm trying to drop an index using migrations.RunSQL but I having the issue that doesn't exist, is there a way to Drop an index only in the case of exist? Something like migrations.RunSQL("DROP INDEX IF EXISTS index_id ON table").
Django 1.8.18
MySQL 5.6
Thank you so much
Since IF EXISTS is not supported in indexing by MySQL, you may want to write your own migration:
def drop_index_if_exists(apps, schema_editor):
# access to the connection since schema_editor.execute does not return the cursor
with schema_editor.connection.cursor() as cursor:
cursor.execute("SHOW INDEX FROM table_name WHERE KEY_NAME = 'index_name'");
exists = int(cursor.fetchone()) > 0
# outside with to close the cursor
if exists:
schema_editor.execute("CREATE INDEX index_name ON ...")
operations = [
migrations.RunPython(drop_index_if_exists)
]
For consistency, you can write a create_index_if_not_exists method to un-apply the migration, and call it:
migrations.RunPython(drop_index_if_exists, create_index_if_not_exists)
Here there is one solution, thank you for idea #alfonso.kim
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations
def drop_index_if_exists_and_create(apps, schema_editor):
# access to the connection since schema_editor.execute does not return the cursor
with schema_editor.connection.cursor() as cursor:
cursor.execute("SHOW INDEX FROM table_name WHERE KEY_NAME = 'index_name'")
exists = True if cursor.fetchone() else False
# outside with to close the cursor
if exists:
schema_editor.execute("DROP INDEX index_name ON table_name")
schema_editor.execute("CREATE INDEX index_name ON table_name(index_name(191))")
class Migration(migrations.Migration):
dependencies = [
('table_name', ''),
]
operations = [
migrations.RunPython(drop_index_if_exists_and_create)
]
I'm getting a Transaction Error after executing your code:
django.db.transaction.TransactionManagementError: Executing DDL statements while in a transaction on databases that can't perform a rollback is prohibited.
Django 2.5
MySql 5.7

Can rspec change and use an innodb fulltext index in the same test?

I have an oddly specific problem. Let's say I have this table in a Rails project:
create_table "documents", force: true do |t|
t.text "tags"
end
add_index "documents", ["tags"], name: "index_documents_on_tags", type: :fulltext
I have an integration test that creates a few Document instances with varying tag combinations, which the method I'm trying to test should return by way of a fulltext search. Unfortunately, it turns out that InnoDB doesn't rebuild its fulltext indices until the current transaction ends, meaning that my search comes up empty.
If I build the test data in fixtures (e.g. in advance, outside of the transaction that rspec uses for each test) it all works fine, but is there any way for me to tweak the data and run a search against it within the same test?
Tricky but fixable. Bear with me.
Step 1
Add this wonderful helper by #mattias (https://stackoverflow.com/a/7703220/537648)
def without_transactional_fixtures(&block)
self.use_transactional_fixtures = false
before(:all) do
DatabaseCleaner.strategy = :truncation
end
yield
after(:all) do
DatabaseCleaner.strategy = :transaction
end
end
Step 2
Add this before block to your rspec examples
Sample usage:
describe "doing my thing" do
before do
# This will rebuild the indexes. You need it before each example
ActiveRecord::Base.connection.execute("ANALYZE TABLE `searchables`")
end
without_transactional_fixtures do
it "does something without transaction fixtures" do
...
end
end
end
Bonus Step
If you are getting this error:
ActiveRecord::StatementInvalid: Mysql2::Error: SAVEPOINT active_record_1 does not exist: ROLLBACK TO SAVEPOINT active_record_1
Be careful when using FactoryBot/FactoryGirl. Use let! instead of let if you need to create objects to the searchable table.
Example:
describe '.search' do
without_transactional_fixtures do
let! (:campaign0) { create(:campaign, io_number: 'C0-1234-4321', status: 'completed') }
let! (:campaign1) { create(:campaign, io_number: "C1-4321-4321") }
before do
ActiveRecord::Base.connection.execute("ANALYZE TABLE `searchables`")
end
...
Thank you #awaage (https://stackoverflow.com/a/13732210/537648)
I had the same question and didn't find a very good solution. One thing you can do is use a tool like DatabaseCleaner and change your strategy for those tests from "transaction" to "truncation".
I had the same issue, and resolved it by manually create the fulltext index in the test.
Example: for your case
create_table "documents", force: true do |t|
t.text "tags"
end
add_index "documents", ["tags"], name: "index_documents_on_tags", type: :fulltext
in your test:
before do
#index_key = "index_documents_on_tags_#{Time.current.to_i}"
ActiveRecord::Base.connection.execute("CREATE FULLTEXT INDEX #{#index_key} ON documents(tags)")
end
after do
ActiveRecord::Base.connection.execute("ALTER TABLE documents DROP INDEX #{#index_key}")
end

Is there a tool to check database integrity in Django?

The MySQL database powering our Django site has developed some integrity problems; e.g. foreign keys that refer to nonexistent rows. I won't go into how we got into this mess, but I'm now looking at how to fix it.
Basically, I'm looking for a script that scans all models in the Django site, and checks whether all foreign keys and other constraints are correct. Hopefully, the number of problems will be small enough so they can be fixed by hand.
I could code this up myself but I'm hoping that somebody here has a better idea.
I found django-check-constraints but it doesn't quite fit the bill: right now, I don't need something to prevent these problems, but to find them so they can be fixed manually before taking other steps.
Other constraints:
Django 1.1.1 and upgrading has been determined to break things
MySQL 5.0.51 (Debian Lenny), currently with MyISAM tables
Python 2.5, might be upgradable but I'd rather not right now
(Later, we will convert to InnoDB for proper transaction support, and maybe foreign key constraints on the database level, to prevent similar problems in the future. But that's not the topic of this question.)
I whipped up something myself. The management script below should be saved in myapp/management/commands/checkdb.py. Make sure that intermediate directories have an __init__.py file.
Usage: ./manage.py checkdb for a full check; use --exclude app.Model or -e app.Model to exclude the model Model in the app app.
from django.core.management.base import BaseCommand, CommandError
from django.core.management.base import NoArgsCommand
from django.core.exceptions import ObjectDoesNotExist
from django.db import models
from optparse import make_option
from lib.progress import with_progress_meter
def model_name(model):
return '%s.%s' % (model._meta.app_label, model._meta.object_name)
class Command(BaseCommand):
args = '[-e|--exclude app_name.ModelName]'
help = 'Checks constraints in the database and reports violations on stdout'
option_list = NoArgsCommand.option_list + (
make_option('-e', '--exclude', action='append', type='string', dest='exclude'),
)
def handle(self, *args, **options):
# TODO once we're on Django 1.2, write to self.stdout and self.stderr instead of plain print
exclude = options.get('exclude', None) or []
failed_instance_count = 0
failed_model_count = 0
for app in models.get_apps():
for model in models.get_models(app):
if model_name(model) in exclude:
print 'Skipping model %s' % model_name(model)
continue
fail_count = self.check_model(app, model)
if fail_count > 0:
failed_model_count += 1
failed_instance_count += fail_count
print 'Detected %d errors in %d models' % (failed_instance_count, failed_model_count)
def check_model(self, app, model):
meta = model._meta
if meta.proxy:
print 'WARNING: proxy models not currently supported; ignored'
return
# Define all the checks we can do; they return True if they are ok,
# False if not (and print a message to stdout)
def check_foreign_key(model, field):
foreign_model = field.related.parent_model
def check_instance(instance):
try:
# name: name of the attribute containing the model instance (e.g. 'user')
# attname: name of the attribute containing the id (e.g. 'user_id')
getattr(instance, field.name)
return True
except ObjectDoesNotExist:
print '%s with pk %s refers via field %s to nonexistent %s with pk %s' % \
(model_name(model), str(instance.pk), field.name, model_name(foreign_model), getattr(instance, field.attname))
return check_instance
# Make a list of checks to run on each model instance
checks = []
for field in meta.local_fields + meta.local_many_to_many + meta.virtual_fields:
if isinstance(field, models.ForeignKey):
checks.append(check_foreign_key(model, field))
# Run all checks
fail_count = 0
if checks:
for instance in with_progress_meter(model.objects.all(), model.objects.count(), 'Checking model %s ...' % model_name(model)):
for check in checks:
if not check(instance):
fail_count += 1
return fail_count
I'm making this a community wiki because I welcome any and all improvements to my code!
Thomas' answer is great but is now a bit out of date.
I have updated it as a gist to support Django 1.8+.