So Rails 4.2 starts to support fractional seconds for MySQL. However it requires a migration of column change from datetime to datetime(6). Currently we do not want to do this, and just want to ignore fractional seconds as we have always been doing before.
Rails is not smart enough to see that my datetime has precision 0 and change queries accordingly, so a lot of our spec broke. For example we assert to "select all Foo created within the last hour", but values are persisted without milliseconds, but select statements still uses milliseconds, so lots of records won't be selected.
Before Rails 4.2:
where("foo_at >= ?", 1.day.ago) => foo_at >= '2015-11-02 04:48:18'
In Rails 4.2:
where("foo_at >= ?", 1.day.ago) => foo_at >= '2015-11-02 04:48:18.934162'
Is there a global setting to force AR queries to strip out fractional/milliseconds as it has been doing before?
There is no official way to turn it off, but you could overwrite the behavior,
module ActiveRecord
module ConnectionAdapters
class Mysql2Adapter < AbstractMysqlAdapter
def quoted_date(value)
super
end
end
end
end
It is in https://github.com/rails/rails/blob/v4.2.5/activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb#L77-L83
And you could remove it in Rails 5, the commit in rails 5 https://github.com/rails/rails/commit/bb9d20cf3434e20ac6f275e3ad29be42ebec207f should Format the time string according to the precision of the time column
From reading Rails source for 4.2.x branch, I don't see any option to change that.
I see a merged PR, which will see what the database column's precision is, and build the sql datetime string according to that precision. This should eliminate my issue that my spec broke due to mismatch of precision between persisted data and queries. However this is not merged in 4.2.x so it probably will only be available in Rails 5.
Related
I want to be able to find a User record by its created_at timestamp (which in this case is: 2020-08-07 11:30:28.5934908).
But the result is this:
User.where(created_at: '2020-08-07 11:30:28.5934908').first
=> nil
The reason that this (existing) User record is not found seems to become evident in the MySQL query that Rails generated:
User Load (0.4ms) SELECT `users`.* FROM `users` WHERE `users`.`created_at` = '2020-08-07 11:30:28.593490' ORDER BY `users`.`id` ASC LIMIT 1
...where for some reason the last digit 8 is dropped from the timestamp 2020-08-07 11:30:28.5934908 used in the MySQL query.
What is the problem here? Does Rails shorten the timestamp in the query? Or does MySQL do this? How do I solve this?
Based on your comments I think that the main problem is in the conversion of the microseconds-precise time to Float. Float (even though in ruby internally a Double) does not have enough accuracy to fully represent all dates / times with microseconds precision, as is documented in the Time class (although they speak about “nanoseconds”, interestingly). Such conversion then only tries to find the nearest possible representation in Float. Rounding the resulting float number back to 6 digits may work but I’m not sure it’s guaranteed to always work…
Suppose that the real time stored in DB is 2020-08-07 11:30:28.593491. As you’ve noticed, this converts to Float imprecisely:
>> Time.parse('2020-08-07 11:30:28.593491').to_f
=> 1596792628.5934908
The guaranteed method would be to use a Rational number instead, i.e. to_r:
>> Time.parse('2020-08-07 11:30:28.593491').to_r
=> (1596792628593491/1000000)
To reconstruct the Time back from the rational number, you can use Time.at:
>> Time.at(Rational(1596792628593491, 1000000)).usec
=> 593491
Note that the microseconds are fully preserved here.
So, storing a created_at time precisely and using it later to search for a record involves using a Rational number variable instead of Float:
>> user_created_at = User.first.created_at.to_r
=> (1596792628593491/1000000)
>> User.where(created_at: Time.at(user_created_at)).first == User.first
=> true
An alternative approach might be to store both the integer seconds since Epoch (User.first.created_at.to_i) and the nanoseconds fraction (User.first.created_at.usec) separately in two variables. They can be the used in Time.at, too, to reconstruct the time back.
As a sidenote, this has also been discussed in a Rails issue with a similar conclusion.
This isn't a direct answer to your question rather a workaround of sorts. You could query the created_at column in some very small time delta like this.
User.where(created_at: '2020-08-07 11:30:28.593490'...'2020-08-07 11:30:28.593491')
As per #BoraMa's comment (thanks!), the MySQL created_at timestamp must have 6 digits only. Thus, the working query would look like this:
User.where(created_at: '2020-08-07 11:30:28.5934908'.round(6)).first
I am trying to insert a date value in MySql table name person and column name regdate with data type = datetime. I am setting a value e.g. '2019-08-21 20:25:20' but after saving +5:30 hours get added and value which gets stored is '2019-08-22 03:55:20'. Generating the date value using below Java code
Timestamp curDate = Timestamp.valueOf(Instant.now().atZone(ZoneId.of("Asia/Kolkata")).toLocalDateTime());
and then using .setTimestamp(1, curdate); in INSERT query.
I have checked that the timezone of MySql is set to IST (GMT+0530). App Server timezone is also set to IST. But I am not able to understand why +5:30 hours are getting added even if I explictly setting the date value.
I have tried setting timezone in connection string as ?serverTimezone=Asia/Kolkata but didn't work.
But if I run the same code using my local machine connecting same MySql instance, I get no problem and same value gets stored without addition of 5:30 hours. I checked App Server timezone and it is IST.
MySql version - 5.7.17-log
mysql-connector-java - 8.0.15
Am I missing something?
You have a few problems here.
Avoid legacy date-time classes
First of all, you are mixing the terrible legacy date-time classes (java.sql.Timestamp) with the modern java.time classes. Don’t. Use only classes from the java.time packages.
LocalDateTime cannot represent a moment
You are using LocalDateTime to track a moment, which it cannot. By definition, that class does not represent a point on the time line. That class has a date and a time-of-day but intentionally lacks the context of a time zone or offset-from-UTC. Calling toLocalDateTime strips away vital information about zone/offset.
Tools lie
You are likely getting confused by the well-intentioned but unfortunate behavior of many tools to dynamically apply a time zone while generating text to represent the date-time value retrieved from the database. You can avoid this by using Java & JDBC to get the pure unadulterated value from the database.
TIMESTAMP WITH TIME ZONE
You failed to disclose the exact data type of your column in your database. If you are trying to track a moment, use a data type akin to the SQL-standard type TIMESTAMP WITH TIME ZONE. In MySQL 8 that would, apparently, be the TIMESTAMP type according to this doc. (I am a Postgres guy, not a MySQL user.)
In JDBC 4.2 and later, we can exchange java.time objects with the database. So no need to over touch java.sql.Timestamp again.
Unfortunately, the JDBC spec oddly chose to not require support for Instant (a moment in UTC) nor forZonedDateTime(a moment as seen in some particular time zone). The spec does require support for [OffsetDateTime`]2.
Tip: Learn to work in UTC for the most part. Adjust into a time zone only when required by business logic or for presentation to the user.
OffsetDateTime odt = OffsetDateTime.now( ZoneOffset.UTC ) ; // Capture current moment in UTC.
Write to the database via a prepared statement.
myPreparedStatement.setObject( … , odt ) ;
Retrieval.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ;
View that moment through the wall-clock time used by the people of some particular region (a time zone).
ZoneId z = ZoneId.of( "Asia/Kolkata" ) ;
ZonedDateTime zdt = odt.atZoneSameInstant( z ) ;
I'm working with Rails 3.2. My development box is using SQLite3 and my production host is using MySQL. Rails SQLite ActiveRecord connector will not save booleans as 1 or 0 and will only save it as 't' or 'f'. Of course I want DB neutral code but I cannot find any way around the following. I have a user model and a shift model. In this query I need all the shifts (work schedules) and I need to order the results by the related table as well as apply the boolean conditions.
#sh= Shift.find(:all, :include=>:user, :order=>'users.rating DESC', :conditions=>["a1=1 or a1='t'"])
I have also learned about ActiveRecord::Base.connection.quoted_true and quoted_false. I suspect I could change them but that also seems non portable and would probably be silently overridden if I upgrade.
I don't want to test for both 1 and 't' (or 0 and 'f'). Is there any way around doing this besides changing my dev environment to mysql?
You should be able to pass the following as a conditions:
:conditions => [["a1 = ?", true]]
I'm currently working with a rails 3.0.x app that's recently been upgraded from rails 2.x. Recently, it has been discovered that our timestamps in MySQL have changed from being written in PST timezone offset to a UTC offset as a result of this migration.
After examining the problem it's been decided we want to keep all our timestamps in a UTC format going forward. Unfortunately, we need to go through all our old records that have timestamps in PST and convert them to UTC (a somewhat complicated process). I'd rather not re-invent the wheel if at all possible, which leads me to my question:
Has anyone written a utility to handle conversion of PST timestamps to UTC timestamps for a MySQL database?
This is why you always save times in UTC and render them in the user's local time if required. Sorry you had to find out the hard way!
What you could do is make a note of the id values where the transition occurred, and then adjust all timestamps prior to that interval with the CONVERT_TZ() method in MySQL.
That would look something like this for each table:
# List of maximum ID to adjust
max_id = {
'examples' => 100,
}
c = ActiveRecord::Base.connection
c.tables.each do |table|
# Set your specific time-zones as required, PST used as an example here.
timestamp_updates = c.columns(table).select do |col|
col.type == :datetime
end.collect(&:name).collect do |col|
"`#{col}`=CONVERT_TZ(`#{col}`, 'PST', 'UTC')"
end
next if (timestamp_updates.empty?)
c.execute("UPDATE `#{table}` SET #{timestamp_updates.join(',') WHERE id<=%d" % max_id[table])
end
I'm writing a rails application on top of a legacy mysql db which also feeds a PHP production tool. Because of this setup so its not possible for me to change the databases structure.
The problem I'm having is that two table have a "time" attribute (duration) as long as the time is under 24:00:00 rails handles this, but as soon as rails comes across something like 39:00:34 I get this "ArgumentError: argument out of range".
I've looked into this problem and seen how rails handle the time type, and from my understanding it treats it like a datetime, so a value of 39:00:34 would throw this error.
I need some way of mapping / or changing the type cast so I don't get this error. Reading the value as a string would also be fine.
Any ideas would be most appreciated.
Cheers
I'm not familiar with Rails so there can be a clean, native solution to this, but if all else fails, one workaround might be writing into a VARCHAR field, then running a 2nd query to copy it over into a TIME field within mySQL:
INSERT INTO tablename (name, stringfield)
VALUES ("My Record", "999:02:02");
UPDATE tablename SET datefield = CAST(stringfield as TIME)
WHERE id = LAST_INSERT_ID();