Ruby DBI fetch can't handle all-zero dates in MySQL - mysql

Question

I'm trying to access our database with some fairly simple CGI/Ruby and DBI:
#!/usr/bin/ruby -w
require "dbi"
dbh = DBI.connect("dbi:Mysql:my:mydb", "XXXX", "XXXX")
...
query = "select ETA from mydb.mytable where ID = #{someval}"
rows = dbh.execute(query)
while row = rows.fetch() do
# Do some stuff
...
end
That works fine most of the time, but I hit a record which broke it with the error:
/usr/lib/ruby/vendor_ruby/dbd/Mysql.rb:120:in `parse': invalid date (ArgumentError)
from /usr/lib/ruby/vendor_ruby/dbd/Mysql.rb:120:in `parse'
from /usr/lib/ruby/vendor_ruby/dbi/row.rb:66:in `block in convert_types'
from /usr/lib/ruby/vendor_ruby/dbi/row.rb:65:in `each'
from /usr/lib/ruby/vendor_ruby/dbi/row.rb:65:in `each_with_index'
from /usr/lib/ruby/vendor_ruby/dbi/row.rb:65:in `convert_types'
from /usr/lib/ruby/vendor_ruby/dbi/row.rb:75:in `set_values'
from /usr/lib/ruby/vendor_ruby/dbi/handles/statement.rb:226:in `fetch'
from /usr/lib/cgi-bin/test:39:in `block in <main>'
from /usr/lib/cgi-bin/test:36:in `each'
from /usr/lib/cgi-bin/test:36:in `<main>'
After a bit of detective work I found that it had a date of 0000-00-00 which fetch() doesn't like. Undefined dates are OK, and DBI in Perl can handle all zero dates, it's just DBI in Ruby.
I can fix the database, and I'll try to get the app which wrote the value to the database fixed too, but I think that my Ruby should be resilient to such things. Is there a way to work around this, maybe using rescue somehow?

Answer 1

This is the solution I came up with:
query = "select ETA from mydb.mytable where ID = #{someval}"
rows = dbh.execute(query)
begin
while row = rows.fetch() do
# Do some stuff
...
end
rescue Exception => e
puts "#{e}<br>\n"
retry
end
This works quite nicely as though the retry is starting a new while loop, rows maintains its state so the fetch resumes on the next record.
The only problem is that it's hard to identify the bad record(s). To fix that I issued more queries without the offending field. My somewhat ugly solution:
results = Hash.new
hit_error = false
query = "select ETA,UNIQUE_ID from mydb.mytable where ID = #{someval}"
rows = dbh.execute(query)
begin
while row = rows.fetch() do
# Do some stuff
...
results[row[1]] = row[0]
end
rescue Exception => e
hit_error = true
retry
end
if hit_error
query = "select UNIQUE_ID from mydb.mytable where ID = #{someval}"
rows = dbh.execute(query)
while row = rows.fetch() do
id = row[0]
unless results.has_key?(id)
begin
query = "select ETA for mydb.mytable where UNIQUE_ID = #{id} limit 1"
error = dbh.execute(query)
error.fetch() # expect this to hit same error as before
puts "Unexpected success for UNIQUE_ID #{id}<br>\n"
rescue Exception => e
puts "#{e} at UNIQUE_ID #{id}<br>\n"
end
end
end
end
Finally I'm not sure how valid it is to use DBI/Ruby, it seems deprecated. Same for MySQL/Ruby. I also tried Sequel with little success.

Answer 2

An alternative solution is to use fetch_hash. This fetches all data as strings, which is a little awkward as you need to convert to dates, integers, etc., but does give you the opportunity of trapping the error on the explicit conversion. This also makes it much easier to identify the bad record(s):
query = "select ETA from mydb.mytable where ID = #{someval}"
rows = dbh.execute(query)
while row = rows.fetch_hash() do
begin
eta = Date.parse row["ETA"]
# Do something with the ETA
...
rescue Exception => e
puts "Error parsing date '#{row["ETA"]}': #{e}<br>\n"
# Make do without the ETA
...
end
end

Answer 3

Sequel can handle invalid dates if you use the mysql adapter:
DB = Sequel.connect('mysql://user:password#host/database')
DB.convert_invalid_date_time = :string # or nil
DB.get(Sequel.cast('0000-00-00', Date))
# => "0000-00-00"