I have this weird situation:
When I do on rails console Time.now or Time.zone.now I get the same values (suppose they run at the sime time: 2014-06-05 23:38:06 -0300)
But when I use Time.now in a query like: Match.where("datetime = ?", Time.now) it returns the time 3 hours ahead!
.to_sql output:
SELECT `matches`.* FROM `matches` WHERE (datetime = '2014-06-06 02:38:06')
any thoughts on that?
Rails 4
Mysql 5.5
Those are the same times. One has a UTC offset of -3, the other is in UTC time and doesn't have the offset.
The time you are seeing in sql query is UTC time. The reason you have Time.now and Time.zone.now return the same time in rails console is because you have both your system and your rails application in the same time zone. It is Time.now that returns the system time based on the system time zone and Time.zone.now that returns the application's time zone aware time.
There are couple of options that can be set as far as the timezone used in the query is concerned, you can either set it to :utc which is the default or set it to :local. You have utc configured and this is the default. :local sets the timezone to server's time zone. This setting is set in config/application.rb:
config.active_record.default_timezone = :utc
The second portion - rails application's time zone can also be set in config/application.rb:
config.time_zone = 'Brasilia'
With this setup, when you use time zone aware classes e.g. Time.zone.now to retrieve time in your application they will be using this configured timezone. Also a point to note with the setting config.time_zone would be to use timezone aware classes in your application to ensure consistent translation of time zone regardless of environment.
Related
I have a question regarding timezone for timestamps.
How the timezone in app.php is used? I realized that the behavior is different if I create a timestamp using Carbon or if I query the value (timestamp) from the DB.
Note: My MySQL use system's timezone which is GMT+8 or Asia/Kuala_Lumpur.
Example #1
Set timezone="UTC" in app.php
Create a Carbon instance
>>> new Carbon\Carbon;
=> Carbon\Carbon #1572404830 {#3496
date: 2019-10-30 03:07:10.625282 UTC (+00:00),
}
>>>
Example #2
Set timezone="Asia/Kuala_Lumpur" in app.php
Create a Carbon instance
>>> new Carbon\Carbon;
=> Carbon\Carbon #1572404816 {#3520
date: 2019-10-30 11:06:56.316851 Asia/Kuala_Lumpur (+08:00),
}
For example #1 and #2, this is for me, expected. You got different value based on the timezone. Things got a little weirder (at least for me), when we query a timestamp from the DB.
Example #3
Set timezone="UTC" in app.php
Query from DB
>>> RefCyberCity::whereDataSource('ccms')->take(1)->first()->updated_at
=> Illuminate\Support\Carbon #1572083605 {#3531
date: 2019-10-26 09:53:25.0 UTC (+00:00),
}
Example #4
Set timezone="Asia/Kuala_Lumpur" in app.php
Query from DB
>>> RefCyberCity::whereDataSource('ccms')->take(1)->first()->updated_at
[!] Aliasing 'RefCyberCity' to 'App\RefCyberCity' for this Tinker session.
=> Illuminate\Support\Carbon #1572054805 {#3491
date: 2019-10-26 09:53:25.0 Asia/Kuala_Lumpur (+08:00),
}
We can see that both output 2019-10-26 09:53:25.0 but the timezone is different.
Dates coming from the database are effectively a string, and although MySQL has its own timezone information, that is not reflected in a date string without a timezone in it.
The Laravel DB drivers are asuming that it was written as the timezone that’s set in the app config. If you were to change your timezone mid-project, this would throw out all your dates.
My personal advice on timezones is to keep everything as UTC on your application and servers, then only change the timezone at the last possible moment (such as in the view), based on the user’s preference. This also allows you to output the UTC timestamp and have JavaScript update it to the user’s locale. Timezones are very hard, and if your country has daylight savings, timezones become even more confusing as Europe/London for example shifts timezone depending on the year.
I am trying to insert a date value in MySql table name person and column name regdate with data type = datetime. I am setting a value e.g. '2019-08-21 20:25:20' but after saving +5:30 hours get added and value which gets stored is '2019-08-22 03:55:20'. Generating the date value using below Java code
Timestamp curDate = Timestamp.valueOf(Instant.now().atZone(ZoneId.of("Asia/Kolkata")).toLocalDateTime());
and then using .setTimestamp(1, curdate); in INSERT query.
I have checked that the timezone of MySql is set to IST (GMT+0530). App Server timezone is also set to IST. But I am not able to understand why +5:30 hours are getting added even if I explictly setting the date value.
I have tried setting timezone in connection string as ?serverTimezone=Asia/Kolkata but didn't work.
But if I run the same code using my local machine connecting same MySql instance, I get no problem and same value gets stored without addition of 5:30 hours. I checked App Server timezone and it is IST.
MySql version - 5.7.17-log
mysql-connector-java - 8.0.15
Am I missing something?
You have a few problems here.
Avoid legacy date-time classes
First of all, you are mixing the terrible legacy date-time classes (java.sql.Timestamp) with the modern java.time classes. Don’t. Use only classes from the java.time packages.
LocalDateTime cannot represent a moment
You are using LocalDateTime to track a moment, which it cannot. By definition, that class does not represent a point on the time line. That class has a date and a time-of-day but intentionally lacks the context of a time zone or offset-from-UTC. Calling toLocalDateTime strips away vital information about zone/offset.
Tools lie
You are likely getting confused by the well-intentioned but unfortunate behavior of many tools to dynamically apply a time zone while generating text to represent the date-time value retrieved from the database. You can avoid this by using Java & JDBC to get the pure unadulterated value from the database.
TIMESTAMP WITH TIME ZONE
You failed to disclose the exact data type of your column in your database. If you are trying to track a moment, use a data type akin to the SQL-standard type TIMESTAMP WITH TIME ZONE. In MySQL 8 that would, apparently, be the TIMESTAMP type according to this doc. (I am a Postgres guy, not a MySQL user.)
In JDBC 4.2 and later, we can exchange java.time objects with the database. So no need to over touch java.sql.Timestamp again.
Unfortunately, the JDBC spec oddly chose to not require support for Instant (a moment in UTC) nor forZonedDateTime(a moment as seen in some particular time zone). The spec does require support for [OffsetDateTime`]2.
Tip: Learn to work in UTC for the most part. Adjust into a time zone only when required by business logic or for presentation to the user.
OffsetDateTime odt = OffsetDateTime.now( ZoneOffset.UTC ) ; // Capture current moment in UTC.
Write to the database via a prepared statement.
myPreparedStatement.setObject( … , odt ) ;
Retrieval.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ;
View that moment through the wall-clock time used by the people of some particular region (a time zone).
ZoneId z = ZoneId.of( "Asia/Kolkata" ) ;
ZonedDateTime zdt = odt.atZoneSameInstant( z ) ;
There's lots of stuff on the internet about converting MySQL timestamps, how it works etc. But what I need is the opposite: knowing that MySQL stores every datetime data as UTC in TIMESTAMP fields, all I want is to direclty retrieve the stored UTC value without MySQL messing around the datetime with system/server/connection timezones.
You see, we've built a simple node.js feeder which reads from several third-part MySQL databases (so I can't change their timezone settings) and save the gathered data to a Elasticsearch, as a "denormalization process". As the original data comes from different timezones, I need to store them in UTC, so I can easily coordinate further GETs and aggregations.
I know I can set the connection timezone on the fly and I know I can change every timestamp field fetched in my node application, but since MySQL engine already stores timestamps in UTC, why should I add any other step if I could simply get it directly, without converting functions or costly data processings?
In a nutshell, I'd like to know: is there a way to bypass MySQL automatic GMT conversion?
MySQL provides a UNIX_TIMESTAMP function which returns a raw integer value. And that isn't subject to timezone conversions at all.
SELECT UNIX_TIMESTAMP( timestamp_col ) ...
But that returns a raw integer, not a datetime. The client would need to do the conversion into a "datetime" type object, if that's needed.
Another option would be to use the CONVERT_TZ function to convert to UTC from the session time_zone setting.
SELECT CONVERT_TZ( timestamp_col, ##session.time_zone,'+00:00')
But, that doesn't really "bypass" timezone conversion. One downside of this approach is if the session time_zone is affected by daylight saving time changes, there's ambiguity with a one hour period each year when the clock "falls back" one hour. e.g. Sunday, Nov 1 2015 2AM CDT transition to Sunday Nov 1 2015 1AM CST. (Converting back from UTC, if we get 1:30 AM in the session time_zone, we don't "know" if that's CDT or CST. And the conversion back to UTC doesn't know which it is either.)
Another option (which I think you already mentioned) is changing the session time_zone to UTC. Then you could just return the timestamp_col value as UTC. You could save the current time_zone setting, and set it back when you are done, e.g.
SET #save_session_time_zone := ##session.time_zone ;
SET time_zone = '+00:00' ;
SELECT timestamp_col ...
SET time_zone = #save_session_time_zone ;
But your client Connector might do some not-so-helpful conversions when the time_zone of the MySQL database session doesn't match the time_zone of the client, like the funky shenanigans the JDBC driver (MySQL Connector/J) does. (That concern isn't limited to returning UTC; that's a concern whenever the time_zone of the client doesn't match the time_zone of the database session.)
It looks like there's no way to get the original UTC value from a MySQL field; every single function uses the timezone setting, be that SYSTEM or any other you configure.
The way MySQL forces you to use a date conversion is, at least, very constraining. For example, say you have a MySQL server set to a timezone with GMT -03:00 and GMT/DST -02:00 and you store a datetime like '2016-07-01 10:00:00'. If you select this value after the DST has ended, you'll get '2016-07-01 09:00:00'.
You can't tell what time it is for sure unless you store the GMT offset separately or you previously know what timezone the server was when it was stored.
We used the second approach. We saved the server timezone and used it to calculate the offset and return an ISO datetime, so future calculations can be made easily.
DROP FUNCTION IF EXISTS `iso_datetime`;;
CREATE FUNCTION `iso_datetime` (
p_datetime TIMESTAMP
) RETURNS VARCHAR(25)
READS SQL DATA
BEGIN
DECLARE _timezone VARCHAR(255) DEFAULT NULL;
DECLARE _offset VARCHAR(6) DEFAULT NULL;
SET _timezone = (SELECT timezone FROM network);
SET _offset = (SELECT SUBSTRING(TIMEDIFF(p_datetime,CONVERT_TZ(p_datetime, _timezone,'UTC')), 1,6));
RETURN CONCAT(DATE_FORMAT(p_datetime, '%Y-%m-%dT%H:%i:%S'), _offset);
END;
In order to do so, you have to load timezone info into MySQL, so the server can calculate the tz offset of the date for you.
I want to know what is the time zone that is currently set in the MySQL server. I do not have administrator rights to the computer I am using so I've tried the method by checking the registry.
I am doing a table with a timestamp column and I noticed the time stamped is different than the one on my computer's time. Is there any reason for this? How do I check what timezone it is on the MySQL server? How do I change it to match my local/computer's time?
You can set the timezone (if you know your offset) for the session by using
set session time_zone = '+00:00';
and to revert to the system default
set session time_zone 'SYSTEM';
In an SQL timestamp column, SQL automatically converts the time to UTC before storing it, using the session's current time offset. It will be the machine's time offset unless you change it (3). Depending on your server's settings (sql.ini), it may or may not always concert back to the expect timezone. This probably explains the time discrepancy.
To get the current timezone offset, try executing
SELECT ##session.time_zone;
To manually override the SQL timezone for the rest of a particular session, execute the following, replacing 00:00 with your desired offset:
SET ##session.time_zone = "+00:00";
Have a look at the system_time_zone system variable.
This may help:
http://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html
You can set the system time zone for MySQL Server at startup with the --timezone=timezone_name option to mysqld_safe. You can also set it by setting the TZ environment variable before you start mysqld. The permissible values for --timezone or TZ are system dependent. Consult your operating system documentation to see what values are acceptable.
You can convert a given timestamp to UTC (or any other TZ you want) with CONVERT_TZ
SELECT CONVERT_TZ(NOW(),##session.time_zone,'GMT');
Note that I use NOW() as simple demonstration, you would put in the timestamp you wanted to convert.
By the same token, you could convert a timestamp in your local TZ to the system
SELECT CONVERT_TZ($timestamp,'Your Time Zone' ##session.time_zone);
To check your shared server
<?php
echo date_default_timezone_get();
?>
To change
<?php
date_default_timezone_set("Africa/Addis_Ababa");
echo date_default_timezone_get();
?>
I'm currently working with a rails 3.0.x app that's recently been upgraded from rails 2.x. Recently, it has been discovered that our timestamps in MySQL have changed from being written in PST timezone offset to a UTC offset as a result of this migration.
After examining the problem it's been decided we want to keep all our timestamps in a UTC format going forward. Unfortunately, we need to go through all our old records that have timestamps in PST and convert them to UTC (a somewhat complicated process). I'd rather not re-invent the wheel if at all possible, which leads me to my question:
Has anyone written a utility to handle conversion of PST timestamps to UTC timestamps for a MySQL database?
This is why you always save times in UTC and render them in the user's local time if required. Sorry you had to find out the hard way!
What you could do is make a note of the id values where the transition occurred, and then adjust all timestamps prior to that interval with the CONVERT_TZ() method in MySQL.
That would look something like this for each table:
# List of maximum ID to adjust
max_id = {
'examples' => 100,
}
c = ActiveRecord::Base.connection
c.tables.each do |table|
# Set your specific time-zones as required, PST used as an example here.
timestamp_updates = c.columns(table).select do |col|
col.type == :datetime
end.collect(&:name).collect do |col|
"`#{col}`=CONVERT_TZ(`#{col}`, 'PST', 'UTC')"
end
next if (timestamp_updates.empty?)
c.execute("UPDATE `#{table}` SET #{timestamp_updates.join(',') WHERE id<=%d" % max_id[table])
end