I'm fairly new to RSpec and have been trying to create some tests for my website, on which a user can post a reservation to the website, which is then saved to our database. I've been trying, using Rspec and Capybara, to simulate a user posting a reservation to the website. We have an existing test database, and at the end of the Rspec test want the new reservation to be written to the database, and not removed at the end of the Rspec test.
One of two things happens when we run the code: either it "works" but the new reservation can't be found in the database, or we get this error:
Failure/Error: Unable to find matching line from backtrace
ActiveRecord::StatementInvalid:
Mysql2::Error: This connection is in use by: #<Thread:0x007fb421fd6218 sleep>: SELECT `users`.* FROM `users` WHERE `users`.`id` = 6 ORDER BY `users`.`id` ASC LIMIT 1
# ./app/controllers/application_controller.rb:95:in `pass_login_status_to_js'
# ./app/middleware/search_suggestions.rb:12:in `call'
Why would this be happening? I realize that Capybara isn't generally meant to be making permanent changes to a database; is there a different program/gem you recommend?
I currently have config.use_transactional_fixtures = false, and also have added the following on the recommendation of a few websites:
class ActiveRecord::Base
mattr_accessor :shared_connection
##shared_connection = nil
def self.connection
##shared_connection || retrieve_connection
end
end
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection
To reiterate, I do want Capybara to be writing to my database (we use SQL). What can I do differently? Does it have something to do with database cleaner?
Yes, it has everything to do with database_cleaner. If you have it setup properly, it will clean your database between scenarios, to keep the tests isolated.
There are a few ways to do what you want:
You can explicitly tell database_cleaner not to clean certain tables between scenarios:
DatabaseCleaner.strategy = :transaction, {except: [:countries, :states]}
DatabaseCleaner.clean_with(:truncation, {except: [:countries, :states]})
You can add your code to a before(:each) or before(:all) block
You can add your data to one or many fixtures
There are only a few cases where you should share data between scenarios (ie. countries, states tables, which are good candidates for #3)
In any other case, I advise against sharing data between scenarios.
Related
I want to update all of a column in a table with over 2.2 million rows where the attribute is set to null. There is a Users table and a Posts table. Even though there is a column for num_posts in User, only about 70,000 users have that number populated; otherwise I have to query the db like so:
#num_posts = #user.posts.count
I want to use a migration to update the attributes and I'm not sure whether or not it's the best way to do it. Here is my migration file:
class UpdateNilPostCountInUsers < ActiveRecord::Migration
def up
nil_count = User.select(:id).where("num_posts IS NULL")
nil_count.each do |user|
user.update_attribute :num_posts, user.posts.count
end
end
def down
end
end
In my console, I ran a query on the first 10 rows where num_posts was null, and then used puts for each user.posts.count . The total time was 85.3ms for 10 rows, for an avg of 8.53ms. 8.53ms*2.2million rows is about 5.25 hours, and that's without updating any attributes. How do I know if my migration is running as expected? Is there a way to log to the console %complete? I really don't want to wait 5+ hours to find out it didn't do anything. Much appreciated.
EDIT:
Per Max's comment below, I abandoned the migration route and used find_each to solve the problem in batches. I solved the problem by writing the following code in the User model, which I successfully ran from the Rails console:
def self.update_post_count
nil_count = User.select(:id).where("num_posts IS NULL")
nil_count.find_each { |user|
user.update_column(:num_posts, user.posts.count) if user.posts
}
end
Thanks again for the help everyone!
desc 'Update User post cache counter'
task :update_cache_counter => :environment do
users = User.joins('LEFT OUTER JOIN "posts" ON "posts.user_id" = "users.id"')
.select('"users.id", "posts.id", COUNT("posts.id") AS "p_count"')
.where('"num_posts" IS NULL')
puts "Updating user post counts:"
users.find_each do |user|
print '.'
user.update_attribute(:num_posts, user.p_count)
end
end
First off don't use a migration for what is essentially a maintenance task. Migrations should mainly alter the schema of your database. Especially if it is long running like in this case and may fail midway resulting in a botched migration and problems with the database state.
Then you need to address the fact that calling user.posts is causing a N+1 query and you instead should join the posts table and select a count.
And without using batches you are likely to exhaust the servers memory quickly.
You can use update_all and subquery to do this.
sub_query = 'SELECT count(*) FROM `posts` WHERE `posts`.`user_id` = `users`.`id`'
User.where('num_posts IS NULL').update_all('num_posts = (#{sub_query})')
It will take only seconds instead of hours.
If so, you may not have to find a way to log something.
I have coded a Ruby IRC bot which is on github (/ninjex/rubot) which is having some conflicting output with MySQL on a dedicated server I just purchased.
Firstly we have the connection to the database in the MySQL folder (in .gitignore) which looks similar to the following code block.
#con = Mysql.new('localhost', 'root', 'pword', 'db_name')
Then we have an actual function to query the database
def db_query
que = get_message # Grabs query from user i.e,./db_query SELECT * FROM words
results = #con.query(que) # Send query through the connection i.e, #con.query("SELECT * FROM WORDS")
results.each {|x| chan_send(x)} # For each row returned, send it to the channel via
end
On my local machine, when running the command:
./db_query SELECT amount, user from words WHERE user = 'Bob' and word = 'hello'
I receive the output in IRC in an Array like fashion: ["17", "Bob"] Where 17 is amount and Bob is the user.
However, using this same function on my dedicated server results in an output like: 17Bob I have attempted many changes in the code, as well as try to parse the data into it's own variable, however it seems that 17Bob is coming out as a single variable, making it impossible to parse into something like an array, which I could then use to send the data correctly.
This seems odd to me on both my local machine and the dedicated server, as I was expecting the output to first send 17 to the IRC and then Bob like:
17
Bob
For all the functions and source you can check my github /Ninjex/rubot, however you may need to install some gems.
A few notes:
Make sure you are sanitizing query via get_message. Or you are opening yourself up to some serious security problems.
Ensure you are using the same versions of the mysql gem, ruby and MySql. Differences in any of these may alter the expected output.
If you are at your wits end and are unable to resolve the underlying issue, you can always send a custom delimiter and use it to split. Unfortunately, it will muck up the case that is actually working and will need to be stripped out.
Here's how I would approach debugging the issue on the dedicated machine:
def db_query
que = get_sanitized_message
results = #con.query(que)
require 'pry'
binding.pry
results.each {|x| chan_send(x)}
end
Add the pry gem to your Gemfile, or gem install pry.
Update your code to use pry: see above
This will open up a pry console when the binding.pry line is hit and you can interrogate almost everything in your running application.
I would take a look at results and see if it's an array. Just type results in the console and it will print out the value. Also type out results.class. It's possible that query is returning some special result set object that is not an array, but that has a method to access the result array.
If results is an array, then the issue is most likely in chan_send. Perhaps it needs to be using something like puts vs print to ensure there's a new line after each message. Is it possible that you have different versions of your codebase deployed? I would also add a sleep 1 within the each block to ensure that this is not related to your handling of messages arriving at the same time.
So I am building a new app that needs to do some importing of legacy data from an old app. The old apps database is mysql, which you obviously can't use on heroku, but I want to use postgres. Basically I am doing an ETL via activerecord.
Here's what I have so far:
# config/initializers/legacy_database.rb
LEGACY_DATABASE_URL = "mysql://myusername:#{ENV['LEGACY_DATABASE_PASSWORD']}#host/foo1008801154002"
# app/models/legacy.rb
class Legacy < ActiveRecord::Base
establish_connection LEGACY_DATABASE_URL
end
# app/models/legacy/user.rb
class Legacy::User < Legacy
self.table_name = 'users'
end
If I am in the console and I run Legacy::User.count I get back the correct count. However if I try to do something like Legacy::User.first I get the following error:
Legacy::User Load (54.0ms) SELECT `users`.* FROM `users` ORDER BY `users`.`id` DESC LIMIT 1
Mysql::Error: Table 'foo1008801154002.legacies' doesn't exist: SHOW FULL FIELDS FROM `legacies`
ActiveRecord::StatementInvalid: Mysql::Error: Table 'foo1008801154002.legacies' doesn't exist: SHOW FULL FIELDS FROM `legacies`
I'm not sure why rails is adding on the .legacies to the table name, nor am I sure how to fix this. I figure it might be some setting in Legacy.connection
Any advice?
Well it was relatively simple. I had to add the line self.abstract_class = true to my Legacy Base class.
Just a note: if you have mysql2 gem in your bundle then need to put:
# config/initializers/legacy_database.rb
LEGACY_DATABASE_URL = "mysql2://myusername:#{ENV['LEGACY_DATABASE_PASSWORD']}#host/foo1008801154002"
(or load mysql gem in Gemfile)
I'm working on ROR 3 app . I have added the following observer but I dont see any output as expected in the console or log file ( i have tried in both development and production modes)
cmd : rails g observer auditor
models:
class AuditorObserver < ActiveRecord::Observer
observe :excel_file
def after_update(excel_file)
excel_file.logger.info('New contact added!')
AuditTrail.new(execl_file, "UPDATED")
puts "*******************"
logger.info "********************************************"
end
end
application.rb:
config.active_record.observers = :auditor_observer
What am I missing in here? When I change the database (thru Mysql workbench/command line) I don't see any of the above lines getting executed.. neither after_update/after_save. But after_save works if I'm executing a query thru the app itself and do #excel.save
How else are we supposed to update data in DB so that we see the observer working????
When you bypass activerecord by modifying the database directly, you naturally bypass all of the activerecord callbacks.
So the answer is to update the data through the application, or to use database triggers instead.
I have the following code that run on heroku inside a controller that intermittently fails. It's a no-brainer that it should work to me, but I must be missing something.
#artist = Artist.find(params[:artist_id])
The parameters hash looks like this:
{"utf8"=>"������",
"authenticity_token"=>"XXXXXXXXXXXXXXX",
"password"=>"[FILTERED]",
"commit"=>"Download",
"action"=>"show",
"controller"=>"albums",
"artist_id"=>"62",
"id"=>"157"}
The error I get looks like this:
ActiveRecord::StatementInvalid: Mysql::Error: : SELECT `artists`.* FROM `artists` WHERE `artists`.`id` = ? LIMIT 1
notice the WHEREartists.id= ? part of the statement? It's trying to find an ID of QUESTION MARK. Meaning Rails is not passing in the params[:artist_id] which is obviously in the params hash. I'm at complete loss.
I get the same error on different pages trying to select the record in a similar fashion.
My environment: Cedar Stack on Heroku (this only happens on Heroku), Ruby 1.9.3, Rails 3.2.8, files being hosted on Amazon S3 (though I doubt it matters), using the mysql gem (not mysql2, which doesn't work at all), ClearDB MySQL database.
Here's the full trace.
Any help would be tremendously appreciated.
try sql?
If it's just this one statement, and it's causing production problems, can you omit the query generator just for now? In other words, for very short term, just write the SQL yourself. This will buy you a bit of time.
# All on one line:
Artist.find_by_sql
"SELECT `artists`.* FROM `artists`
WHERE `artists`.`id` = #{params[:artist_id].to_i} LIMIT 1"
ARel/MySQL explain?
Rails can help explain what MySQL is trying to do:
Artist.find(params[:artist_id]).explain
http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/
Perhaps you can discover some kind of difference between the queries that are succeeding vs. failing, such as how the explain uses indexes or optimizations.
mysql2 gem?
Can you try changing from the mysql gem to the mysql2 gem? What failure do you get when you switch to the mysql2 gem?
volatility?
Perhaps there's something else changing the params hash on the fly, so you see it when you print it, but it's changed by the time the query runs?
Try assigning the variable as soon as you receive the params:
artist_id = params[:artist_id]
... whatever code here...
#artist = Artist.find(artist_id)
not the params hash?
You wrote "Meaning Rails is not passing in the params[:artist_id] which is obviously in the params hash." I don't think that's the problem-- I expect that you're seeing this because Rails is using the "?" as a placeholder for a prepared statement.
To find out, run the commands suggested by #Mori and compare them; they should be the same.
Article.find(42).to_sql
Article.find(params[:artist_id]).to_sql
prepared statements?
Could be a prepared statement cache problem, when the query is actually executed.
Here's the code that is failing-- and there's a big fat warning.
begin
stmt.execute(*binds.map { |col, val| type_cast(val, col) })
rescue Mysql::Error => e
# Older versions of MySQL leave the prepared statement in a bad
# place when an error occurs. To support older mysql versions, we
# need to close the statement and delete the statement from the
# cache.
stmt.close
#statements.delete sql
raise e
end
Try configuring your database to turn off prepared statements, to see if that makes a difference.
In your ./config/database.yml file:
production:
adapter: mysql
prepared_statements: false
...
bugs with prepared statements?
There may be a problem with Rails ignoring this setting. If you want to know a lot more about it, see this discussion and bug fix by Jeremey Cole and Aaron: https://github.com/rails/rails/pull/7042
Heroku may ignore the setting. Here's a way you can try overriding Heroku by patching the prepared_statements setup: https://github.com/rails/rails/issues/5297
remove the query cache?
Try removing the ActiveRecord QueryCache to see if that makes a difference:
config.middleware.delete ActiveRecord::QueryCache
http://edgeguides.rubyonrails.org/configuring.html#configuring-middle
try postgres?
If you can try Postgres, that could clear it up too. That may not be a long term solution for you, but it would isolate the problem to MySQL.
The MySQL statement is obviously wrong, but the Ruby code you mentioned would not produce it. Something is wrong here, either you use a different Ruby code (maybe one from a before_filter) or pass a different parameter (like params[:artist_id] = "?"). Looks like you use nested resources, something like Artist has_many :albums. Maybe the #artist variable is not initialized correctly in the previous action, so that params[:artist_id] has not the right value?