Why are stored procedures still not supported in Rails (3+)? - mysql

I am familiar with the long standing love-hate relationship between Ruby on Rails, DB(MS)-drivers and Stored Procedures and I have been developing Rails applications since version 2.3.2.
However, every once in a while a situation arises where a SP is simply a better choice than combining data on the (much slower) application level. Specifically, running reports which combines data from multiple tables is usually better suited for a SP.
Why are stored procedures still so poorly integrated into Rails or the MySQL gem. I am currently working on a project with Rails 3.0.10 and MySQL2 gem 0.2.13 but as far as I can see, even the latest Edge Rails and MySQL gem 0.3+ still throw tantrums when you use SPs.
The problem which has been, and still is, is that the database connection is lost after a SP is called.
>> ActiveRecord::Base.connection.execute("CALL stored_proc")
=> #<Mysql::Result:0x103429c90>
>> ActiveRecord::Base.connection.execute("CALL stored_proc")
ActiveRecord::StatementInvalid: Mysql::Error: Commands out of sync;
[...]
>> ActiveRecord::Base.connection.active?
=> false
>> ActiveRecord::Base.connection.reconnect!
=> nil
>> ActiveRecord::Base.connection.execute("CALL proc01")
=> #<Mysql::Result:0x1034102e0>
>> ActiveRecord::Base.connection.active?
=> false
Is this a really difficult problem to tackle, technically, or is this a design choice by Rails?

Stored procedures are supported in rails. The out of of sync error you are getting is because the MULTI_STATEMENTS flag for MySQL is not enabled by default in Rails. This flag allows for procedures to return more than 1 result set.
See here for a code sample on how to enable it: https://gist.github.com/wok/1367987
Stored procedures work out of the box with MS SQL Server.
I have been using stored procedures in almost all of my mySQL and SQL Server based rails projects without any issued.

This is for postgres to execute a stored procedure that returns instances of MyClass.
sql=<<-SQL
select * from my_cool_sp_with_3_parameters(?, ?, ?) as
foo(
column_1 <type1>,
column_2 <type2>
)
SQL
MyClass.find_by_sql([sql, param1, param2, param3]);
Replace the column list inside of foo() with the columns from your model and the stored procedure results. I'm sure this could be made generic by inspecting the columns of the class.

Those who are getting sync errors may have procedures that generate multiple results. You will need to do something like this to handle them:
raise 'You updated Rails. Check this duck punch is still valid' unless Rails.version == "3.2.15"
module ActiveRecord
module ConnectionAdapters
class Mysql2Adapter
def call_stored_procedure(sql)
results = []
results << select_all(sql)
while #connection.more_results?
results << #connection.next_result
end
results
end
end
end
end
Call like this:
ActiveRecord::Base.connection.call_stored_procedure("CALL your_procedure('foo')")

Related

Calling MySQL stored procedure in ROR 4

There are few example out there but non of them are very clarified (or on old version).
I want to call MySQL procedure and check the return status (in rails 4.2). The most common method I saw is to call result = ActiveRecord::Base.connection.execute("call example_proc()"), but in some places people wrote there is prepared method result = ActiveRecord::Base.connection.execute_procedure("Stored Procedure Name", arg1, arg2) (however it didn't compiled).
So what is the correct way to call and get the status for MySQL procedure?
Edit:
And how to send parameters safly, where the first parameter is integer, second string and third boolean?
Rails 4 ActiveRecord::Base doesn't support execute_procedure method, though result = ActiveRecord::Base.connection still works. ie
result = ActiveRecord::Base.connection.execute("call example_proc('#{arg1}','#{arg2}')")
You can try Vishnu approach below
or
You can also try
ActiveRecord::Base.connections.exec_query("call example_proc('#{arg1}','#{arg2}')")
here is the document
In general, you should be able to call stored procedures in a regular where or select method for a given model:
YourModel.where("YOUR_PROC(?, ?)", var1, var2)
As for your comment "Bottom line I want the most correct approach with procedure validation afterwards (for warnings and errors)", I guess it always depends on what you actually want to implement and how readable you want your code to be.
For example, if you want to return rows of YourModel attributes, then it probably would be better if you use the above statement with where method. On the other hand, if you write some sql adapter then you might want to go down to the ActiveRecord::Base.connection.execute level.
BTW, there is something about stored proc performance that should be mentioned here. In several databases, database does stored proc optimization on the first run of the stored proc. However, the parameters that you pass to that first run might not be those that will be running on it more frequently later on. As a result, your stored-proc might be auto-optimized in a "none-optimal" way for your case. It may or may not happen this way, but it is something that you should consider while using stored procs with dynamic params.
I believe you have tried many other solutions and got some or other errors mostly "out of sync" or "closed connection" errors. These errors occur every SECOND time you try to execute the queries. We need to workaround like the connection is new every time to overcome this. Here is my solution that didn't throw any errors.
#checkout a connection for Model
conn = ModelName.connection_pool.checkout
#use the new connection to execute the query
#records = conn.execute("call proc_name('params')")
#checkout the connection
ModelName.connection_pool.checkin(conn)
The other approaches failed for me, possibly because ActiveRecord connections are automatically handled to checkout and checking for each thread. When our method tries to checkout a connection just to execute the SP, it might conflict since there will be an active connection just when the method started.
So the idea is to manually #checkout a connection for the model instead of for thread/function from the pool and #checkin once the work is done. This worked great for me.

Ruby MySQL output conflicting on different servers

I have coded a Ruby IRC bot which is on github (/ninjex/rubot) which is having some conflicting output with MySQL on a dedicated server I just purchased.
Firstly we have the connection to the database in the MySQL folder (in .gitignore) which looks similar to the following code block.
#con = Mysql.new('localhost', 'root', 'pword', 'db_name')
Then we have an actual function to query the database
def db_query
que = get_message # Grabs query from user i.e,./db_query SELECT * FROM words
results = #con.query(que) # Send query through the connection i.e, #con.query("SELECT * FROM WORDS")
results.each {|x| chan_send(x)} # For each row returned, send it to the channel via
end
On my local machine, when running the command:
./db_query SELECT amount, user from words WHERE user = 'Bob' and word = 'hello'
I receive the output in IRC in an Array like fashion: ["17", "Bob"] Where 17 is amount and Bob is the user.
However, using this same function on my dedicated server results in an output like: 17Bob I have attempted many changes in the code, as well as try to parse the data into it's own variable, however it seems that 17Bob is coming out as a single variable, making it impossible to parse into something like an array, which I could then use to send the data correctly.
This seems odd to me on both my local machine and the dedicated server, as I was expecting the output to first send 17 to the IRC and then Bob like:
17
Bob
For all the functions and source you can check my github /Ninjex/rubot, however you may need to install some gems.
A few notes:
Make sure you are sanitizing query via get_message. Or you are opening yourself up to some serious security problems.
Ensure you are using the same versions of the mysql gem, ruby and MySql. Differences in any of these may alter the expected output.
If you are at your wits end and are unable to resolve the underlying issue, you can always send a custom delimiter and use it to split. Unfortunately, it will muck up the case that is actually working and will need to be stripped out.
Here's how I would approach debugging the issue on the dedicated machine:
def db_query
que = get_sanitized_message
results = #con.query(que)
require 'pry'
binding.pry
results.each {|x| chan_send(x)}
end
Add the pry gem to your Gemfile, or gem install pry.
Update your code to use pry: see above
This will open up a pry console when the binding.pry line is hit and you can interrogate almost everything in your running application.
I would take a look at results and see if it's an array. Just type results in the console and it will print out the value. Also type out results.class. It's possible that query is returning some special result set object that is not an array, but that has a method to access the result array.
If results is an array, then the issue is most likely in chan_send. Perhaps it needs to be using something like puts vs print to ensure there's a new line after each message. Is it possible that you have different versions of your codebase deployed? I would also add a sleep 1 within the each block to ensure that this is not related to your handling of messages arriving at the same time.

How does Rails build a MySQL statement?

I have the following code that run on heroku inside a controller that intermittently fails. It's a no-brainer that it should work to me, but I must be missing something.
#artist = Artist.find(params[:artist_id])
The parameters hash looks like this:
{"utf8"=>"������",
"authenticity_token"=>"XXXXXXXXXXXXXXX",
"password"=>"[FILTERED]",
"commit"=>"Download",
"action"=>"show",
"controller"=>"albums",
"artist_id"=>"62",
"id"=>"157"}
The error I get looks like this:
ActiveRecord::StatementInvalid: Mysql::Error: : SELECT `artists`.* FROM `artists` WHERE `artists`.`id` = ? LIMIT 1
notice the WHEREartists.id= ? part of the statement? It's trying to find an ID of QUESTION MARK. Meaning Rails is not passing in the params[:artist_id] which is obviously in the params hash. I'm at complete loss.
I get the same error on different pages trying to select the record in a similar fashion.
My environment: Cedar Stack on Heroku (this only happens on Heroku), Ruby 1.9.3, Rails 3.2.8, files being hosted on Amazon S3 (though I doubt it matters), using the mysql gem (not mysql2, which doesn't work at all), ClearDB MySQL database.
Here's the full trace.
Any help would be tremendously appreciated.
try sql?
If it's just this one statement, and it's causing production problems, can you omit the query generator just for now? In other words, for very short term, just write the SQL yourself. This will buy you a bit of time.
# All on one line:
Artist.find_by_sql
"SELECT `artists`.* FROM `artists`
WHERE `artists`.`id` = #{params[:artist_id].to_i} LIMIT 1"
ARel/MySQL explain?
Rails can help explain what MySQL is trying to do:
Artist.find(params[:artist_id]).explain
http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/
Perhaps you can discover some kind of difference between the queries that are succeeding vs. failing, such as how the explain uses indexes or optimizations.
mysql2 gem?
Can you try changing from the mysql gem to the mysql2 gem? What failure do you get when you switch to the mysql2 gem?
volatility?
Perhaps there's something else changing the params hash on the fly, so you see it when you print it, but it's changed by the time the query runs?
Try assigning the variable as soon as you receive the params:
artist_id = params[:artist_id]
... whatever code here...
#artist = Artist.find(artist_id)
not the params hash?
You wrote "Meaning Rails is not passing in the params[:artist_id] which is obviously in the params hash." I don't think that's the problem-- I expect that you're seeing this because Rails is using the "?" as a placeholder for a prepared statement.
To find out, run the commands suggested by #Mori and compare them; they should be the same.
Article.find(42).to_sql
Article.find(params[:artist_id]).to_sql
prepared statements?
Could be a prepared statement cache problem, when the query is actually executed.
Here's the code that is failing-- and there's a big fat warning.
begin
stmt.execute(*binds.map { |col, val| type_cast(val, col) })
rescue Mysql::Error => e
# Older versions of MySQL leave the prepared statement in a bad
# place when an error occurs. To support older mysql versions, we
# need to close the statement and delete the statement from the
# cache.
stmt.close
#statements.delete sql
raise e
end
Try configuring your database to turn off prepared statements, to see if that makes a difference.
In your ./config/database.yml file:
production:
adapter: mysql
prepared_statements: false
...
bugs with prepared statements?
There may be a problem with Rails ignoring this setting. If you want to know a lot more about it, see this discussion and bug fix by Jeremey Cole and Aaron: https://github.com/rails/rails/pull/7042
Heroku may ignore the setting. Here's a way you can try overriding Heroku by patching the prepared_statements setup: https://github.com/rails/rails/issues/5297
remove the query cache?
Try removing the ActiveRecord QueryCache to see if that makes a difference:
config.middleware.delete ActiveRecord::QueryCache
http://edgeguides.rubyonrails.org/configuring.html#configuring-middle
try postgres?
If you can try Postgres, that could clear it up too. That may not be a long term solution for you, but it would isolate the problem to MySQL.
The MySQL statement is obviously wrong, but the Ruby code you mentioned would not produce it. Something is wrong here, either you use a different Ruby code (maybe one from a before_filter) or pass a different parameter (like params[:artist_id] = "?"). Looks like you use nested resources, something like Artist has_many :albums. Maybe the #artist variable is not initialized correctly in the previous action, so that params[:artist_id] has not the right value?

fetching multiple result sets from mysql stored procedure in rails

I've been searching all over for tips on this and have not really had any luck so far. With the mysql2 gem, trying to execute a stored procedure that returns multiple result sets gives me an unable to return results in this context error. I found someone had suggested to use the mysql gem instead (which I can't find an explanation of what's different between the two and what I might encounter by switching), and with that I've had more progress.
Here's what I have so far:
>> db = ActiveRecord::Base.connection.raw_connection
=> #<Mysql:0x1056ae3d8>
>> ActiveRecord::Base.connection.select_all("CALL p_rpt_test('', '');")
=> [{"Header"=>"Client,Project,Type,Due Date,Assigned To"}]
>> db.more_results?
=> true
>> db.next_result
Mysql::Error: Commands out of sync; you can't run this command now
from (irb):3:in `next_result'
from (irb):3
Does anyone know of a way to get this to work, with mysql2 or mysql gems? The app is running rails 3.0.1.
Ok well I have no figured out how to get AR to do this so I've ended up just going low level and using the mysql driver itself, which mostly works...
data = Array.new
db = ActiveRecord::Base.connection.raw_connection
header = db.query("CALL #{self.proc}(#{args});")
header.each {|r| data << r}
if db.next_result
rows = db.store_result
rows.each {|r| data << r}
end
ActiveRecord::Base.connection.reconnect!
It works, but I can't imagine there's not a better way. Also I have to reconnect after this or I get an error on the next query, and I haven't found a way to properly close the session. Oh and I have to use the mysql gem and not mysql2.
Grrrrr.
We can use header.to_hash to get an array of hash, or header.rows to get an array of array.
Follow this http://api.rubyonrails.org/classes/ActiveRecord/Result.html

writing to mysql table with ruby and generating absolutely unique id like youtube?

how can i write to a mysql table with ruby ? run SQL. for example, if the ruby script finishes, write to mysql table saying its done and how can i generate an absolutely unique ID ? kinda like youtube's id.
For the unique ID, it's pretty easy. You should take a look at mysql's auto increment.
For mysql with Ruby, you need to use the mysql gem.
Using it is also pretty easy. You could do something like the following :
require "mysql"
begin
# connect to the MySQL server
dbh = Mysql.real_connect("localhost", "root", "password", "database")
dbh.query("INSERT INTO database (field) VALUES ('value')")
rescue Mysql::Error => e
p "Error code: #{e.errno}"
p "Error message: #{e.error}"
p "Error SQLSTATE: #{e.sqlstate}" if e.respond_to?("sqlstate")
ensure
# disconnect from server
dbh.close if dbh
end
And if you're planning to build something consequent, you might also be interested in using Active Record.
Here's everything you'd want to know about connecting to MySQL from Ruby:
Ruby MySQL API
How to use it
If you're looking for just a unique ID, and you don't mind exposing an obviously incrementing integer to the world, assign an autoincrement integer field to your table. I added the "and you don't mind" part because anyone who looks at your ID will be able to guess the scheme and will likely experiment fetching nearby integers.
If you want a truly random string to use as your ID (which is more Youtube-ish), check out this discussion.