Mysql: Convert DB from local time to UTC - mysql

I need to convert an existing (datetime fields) db from local time ut UTC.
The values are stored ad datetimes on a server with time zone CET (+1) (with summertime +2). When selecting data I use UNIX_TIMESTAMP(), which magically compensates for everything, ie, time zone shift and dst (if i've read the docs right).
I'm moving the db to a new server with UTC as system time.
Simply subtracting -1 H won't work, as summer time is +2.
Any ideas for a clever way to do this? (using sql or some script lang)

First you need to make sure the mysql.time_zone_name table is populated. If it's empty, you can follow the instructions on this page to populate it:
http://dev.mysql.com/doc/refman/5.1/en/time-zone-support.html
It's typically as simple as running a command like this in the shell:
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql
Once that table is populated you can use the CONVERT_TZ() function to update the existing values in the DB:
http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_convert-tz
Here are two examples to show how it converts datetimes from CET to UTC in winter vs summer:
mysql> SELECT CONVERT_TZ('2010-01-22 12:00:00','CET','UTC');
+-----------------------------------------------+
| CONVERT_TZ('2010-01-22 12:00:00','CET','UTC') |
+-----------------------------------------------+
| 2010-01-22 11:00:00 |
+-----------------------------------------------+
1 row in set (0.00 sec)
mysql> SELECT CONVERT_TZ('2010-07-22 12:00:00','CET','UTC');
+-----------------------------------------------+
| CONVERT_TZ('2010-07-22 12:00:00','CET','UTC') |
+-----------------------------------------------+
| 2010-07-22 10:00:00 |
+-----------------------------------------------+
1 row in set (0.00 sec)

It should be noted that the conversion for dates from one timezone to another or to UTC can only be done reliably if the dates are in the past.
Timezone definitions change. They are a human definition of how to deviate from the "sun clock", and those definitions can and do change constantly. So the only valid conversion is for dates in the past, because that will not change anymore.
Any date in the future cannot reliably be converted, because the conversion can only take into account the currently known timezone definition.
Simple example: Let's create a meeting appointment next year in Berlin, Germany. We agree today that we want to meet at 12:00 on July 1st, 2014 at Alexanderplatz. That date would be translated to 10:00 UTC on that day.
Now if some government decides to opt out of daylight saving time in 2014, you'd have a problem deciding whether you should show up at 12:00 local time, or at 11:00 local time, because the conversion back from UTC will result in a different local time.
If you had saved the original date of "2014-07-01 12:00 Europe/Berlin", you will be there at that exact time at noon, like everyone else.

In the original server, you can use one of the following expressions inside an UPDATE query:
CONVERT_TZ(your_datetime_field,'SYSTEM','UTC')
CONVERT_TZ(your_datetime_field,##global.time_zone,'UTC')
Alternatively, in the destination server, if you know the time zone of the original server (for example 'Europe/Berlin') you can use one of the following expressions:
CONVERT_TZ(your_datetime_field,'Europe/Berlin','UTC')
CONVERT_TZ(your_datetime_field,'Europe/Berlin',##global.time_zone)

Related

When using NOW() in MySQL can I be certain that the correct UTC value will be stored in a TIMESTAMP column?

MySQL has two data types that are purpose built for storing date & time values - DATETIME and TIMESTAMP. Neither type stores timezone information, and both have different rules.
A DATETIME column will store the exact date & time value provided in the insertion query. (No converstions and no affordances for time zone)
A TIMESTAMP column will convert the date & time value provided at insertion from the timezone of the connection at insertion time to UTC. On retrieval it will convert the date & time value stored from UTC to the timezone of the retrieval connection.
The timezone of both connections can be explicitly or implicitly set according to these rules.
Now before I get to my question let's look at some of the nuances of handling dates & times when daylight savings is involved. Summarizing the answers on another Stack Overflow question as well as what I understand from the MySQL documentation regarding date/time:
When using a DATETIME column and explicitly specifying a value (ie/ 2009-11-01 01:30:00), the value can be ambiguous. DATETIME performs no conversation and simply stores this exact date/time. Say I'm in New York (which follows a daylight savings time). Both at insertion and retrieval I have no way of indicating/knowing if this value refers to 1:30AM at the with-daylight-savings moment (UTC-4) or 1:30AM at the without-daylight-savings moment (UTC-5).
When using a DATETIME column along with NOW(), NOW() evaluates to the date & time value at the start of query execution (ie/ 2009-11-01 01:30:00) and this value is inserted, with no converstions, into the DATETIME field causing the same exact ambiguity as mentioned above.
When using a TIMESTAMP column and explicitly specifying a value (ie/ 2009-11-01 01:30:00), I again have the same problem as mentioned above. There's no way to specify and no way to know which 1:30am I'm referring to.
Now, here's my question:
Given a MySQL connection that is set to a timezone that includes daylight savings (say America/New York), can I be certain that inserting NOW() into a TIMESTAMP column will cause the correct UTC date & time value to be stored? UTC of course does not observe daylight savings, so the UTC time at the 1:30am New York timezone with-daylight-savings moment is different from the UTC time at the 1:30AM New York timezone without-daylight-savings moment.
More specifically: Is the UTC offset of the connection timezone at the start of query execution what is used to perform the to-UTC/from-UTC conversion when I insert/select from a TIMESTAMP column? Going back to my example, at the 1:30am with-daylight-savings moment (America\New York timezone) I'm at UTC-4 and at the 1:30am without-daylight-savings moment (America\New York timezone) I'm at UTC-5 - so in both these moments, would a different value be stored in a TIMESTAMP field when I explicitly insert 2009-11-01 01:30:00 or implicitly insert this same value by using NOW()? Finally, if I'm within a single MySQL connection that spans both these moments, and I execute two queries (one in the first moment, and a separate one in the second moment), will both queries cause the correct (different) UTC value to be stored?
You might want this for a server:
mysql> SHOW VARIABLES LIKE '%zone%';
+------------------+--------+
| Variable_name | Value |
+------------------+--------+
| system_time_zone | UTC | -- Comes from OS
| time_zone | SYSTEM | -- Probably the 'right' setting
+------------------+--------+
With those settings, SELECT NOW() will deliver UTC time, not local time.
For your personal computer, it is probably better to have system_time_zone equal to something like Pacific Daylight Time (or whatever), to reflect your current location.
No conversion happens when INSERTing or SELECTing a DATE or DATETIME. Think of it as being a picture of the clock.
For TIMESTAMP, whatever you give it is converted to/from UTC. That is, the bits stored in the table are UTC, but you can't see that; you only see the converted date&time based on the two settings above.
I suggest that the best way to get the answer is to create a table with a DATETIME and a TIMESTAMP, set the two settings, then see what happens when storing. Then change the settings and do a SELECT.
As far as I know, you're best off with doing this from your code. If you really have to do it via DB, you could use UTC_TIMESTAMP() which will always give you the current time based on UTC. Relying on your server timezone however is not a good idea in my opinion because the servers are bound to undergo changes as you grow/scale.
On the other hand, specifying the time from code will tend to be more important/accurate for you than letting it hit the DB. (Think latency, delayed inserts and what not).
You can test the time_zone change by using the SET time_zone function :
mysql> create table demo (test timestamp);
mysql> SET time_zone='-06:00';
mysql> insert into demo VALUES(NOW());
mysql> SELECT * FROM demo;
+---------------------+
| test |
+---------------------+
| 2017-05-23 08:55:16 |
+---------------------+
mysql> SET time_zone='+02:00';
mysql> insert into demo VALUES(NOW());
mysql> SELECT * FROM demo;
+---------------------+
| test |
+---------------------+
| 2017-05-23 16:55:16 |
| 2017-05-23 16:55:32 |
+---------------------+
According to those results changing the timezone gives consistent results which prove that times are stored UTC (when using timestamp).
I used this one Not sure if it will helpful or not. Please add the below text in your my.cnf file.
[mysqld_safe]
timezone = UTC

Converting timezone(convert_tz) in mysql [duplicate]

I am trying to set up a database that stores daily alert times as specified by users. For example, the user wants to receive an alert if some criterion is met each day between 7:00 AM and 7:30 AM. In trying to implement this, I need to accommodate daylight saving time. Here's my attempted solution:
Store the users local time zone (in long form, e.g. "US/Eastern") information in one table (say userInfo), and the alarm times in another table (say userAlarms).
When querying the userAlarms table, convert UTC time into the users local time as specified by the tz column stored in the userInfo table via CONVERT_TZ(UTC_TIME(), 'UTC', userInfo.tz).
Question 1. From my understanding, specifying the time zone name (like US/Eastern) should take daylight saving time into account. For example, calling CONVERT_TZ('00:00:00', 'UTC', 'US/EASTERN') on January 1 should yield '19:00:00', but on July 1 the call should yield '20:00:00'. Am I correct?
Question 2. If Q1 is correct, do I need to constantly update MySQL's time zone table to keep the time zone UTC offsets up to date?
Question 3. The sample given in the MySQL documentation SELECT CONVERT_TZ('2004-01-01 12:00:00','GMT','MET') yields "NULL" when run on my server. Could this be caused by not having the time zone tables set-up?
How can I check this?
If this yields null then the TZ tables have not been set up:
SELECT CONVERT_TZ(now(),'US/Eastern','US/Central');
If you do not have the time zone tables set up you could update the hour
offset in the user table and then do:
select utc_timezone() - interval user_timezone_offset_in_hours hour
from userinfo a
where user_id = 999;
You'd still need a way to update the user's time zone however.
If you are writing this for a web application you can get the time zone via javascript, here's an article that describes how (haven't tried this but it looks like it'll work).
A bit of an explanation with respect to 'interval' above...
One of the more trick constructs in MySQL is the use of the INTERVAL
keyword, best shown by example the (numeric value can be an expression or the field value)
select now() today, now() - interval 1 day yesterday;
+---------------------+---------------------+
| today | yesterday |
+---------------------+---------------------+
| 2011-05-26 13:20:55 | 2011-05-25 13:20:55 |
+---------------------+---------------------+
You can add them and subtract them anyway you like, this is why I never
bother with the date/time add/subtract/convert functions
select now() a, now() - interval 1 day + interval 4 hour + interval 8 minute b;
+---------------------+---------------------+
| a | b |
+---------------------+---------------------+
| 2011-05-26 13:24:16 | 2011-05-25 17:32:16 |
+---------------------+---------------------+
You can use negative numbers (should be good for negative time zone offsets)
these are the same:
select now() - interval 1 month a, now() + interval -1 month b;
+---------------------+---------------------+
| a | b |
+---------------------+---------------------+
| 2011-04-26 13:38:05 | 2011-04-26 13:38:05 |
+---------------------+---------------------+
I found this thread helpful and decided to share the doc page for importing this information. I did exactly as instructed below in CentOS and in RHEL and it worked flawlessly. I am now able to use the CONVERT_TZ function with arguments like "GMT" and "US/Eastern".
From the MySQL documentation this is how to import the MySQL time zone table information:
http://dev.mysql.com/tech-resources/articles/4.1/time.html
For all users running MySQL 4.1.3 or later on a Unix-based system (recall this doesn't work on Windows systems yet):
Populate the time zone tables.
To do so, run the mysql_tzinfo_to_sql program, provided with the MySQL distribution. mysql_tzinfo_to_sql reads the operating system time zone files and generates SQL statements from them. The SQL statements are then processed by mysql, to load the time zone tables.
To run mysql_tzinfo_to_sql successfully, one needs to know where the server machine's operating system time zone files are stored; check for a directory with a name similar to /usr/share/zoneinfo. Pass the directory name on the command line to mysql_tzinfo_to_sql, and send the output into the mysql program. Here's an example.
shell> mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql
Note: The above command assumes that "mysql_tzinfo_to_sql" and "mysql" are in your path. If they're not, you'll need to supply the full path to both when running the command, or switch into the mysql bin folder and use a command like so:
shell> ./mysql_tzinfo_to_sql /usr/share/zoneinfo | ./mysql -u root mysql
Q1: Yes, the CONVERT_TZ takes daylight savings time into account for you. This information and the time that DST starts/ends for each time zone is stored in the time_zone_* tables.
Q2: Yes, as stated in the mysql docs, the time zone information changes per the politics of each area. You'll have to update the time_zone_* tables every time a change occurs. Sucks to be IT sometimes, this is one of them.
Q3: These are the 5 timezone tables, query them to see if they have anything in them:
select * from mysql.time_zone_transition_type;
select * from mysql.time_zone_transition;
select * from mysql.time_zone_name;
select * from mysql.time_zone_leap_second;
select * from mysql.time_zone;

Adjusting for timezone and daylight saving time on a per-user basis

I am doing an application where the users store their timezone in an SQL database.
How do I keep track when there is daylight saving time in their locations?
I was thinking of having a column in the users table with the users specified time zone, and then another table where i would match the user time zone with it's corresponding interval displacement regarding UTC
But how do I do a script that automatically changes this off set for places where there is daylight saving time? There are even places when sometimes there is daylight saving and some years there is not.
Should I use an API? If so, which one?
Is my appoach plain wrong?
Thank you
You're reinventing the wheel, here.
MySQL Server has built-in time zone handling capabilities, including the ability to "know" when and where it's daylight time (or not) from your system's time zone tables.
http://dev.mysql.com/doc/refman/5.6/en/time-zone-support.html
The CONVERT_TZ() function, embedded in a query, will convert a datetime from its current time zone (which you specify) into another one (the user's time zone).
How do i update this table when time chages due to daylight savings
You don't. If you have the correct time zone selected for the user, the daylight saving time conversion is handled automatically.
For example, converting a winter-time and a summer-time from UTC into the America/New_York time zone (Eastern Standard/Eastern Daylight). Note the 5-hour (winter) vs 4-hour (summer) offset:
mysql> select convert_tz('2015-01-01 00:00:00','UTC','America/New_York');
+------------------------------------------------------------+
| convert_tz('2015-01-01 00:00:00','UTC','America/New_York') |
+------------------------------------------------------------+
| 2014-12-31 19:00:00 |
+------------------------------------------------------------+
1 row in set (0.01 sec)
mysql> select convert_tz('2015-06-01 00:00:00','UTC','America/New_York');
+------------------------------------------------------------+
| convert_tz('2015-06-01 00:00:00','UTC','America/New_York') |
+------------------------------------------------------------+
| 2015-05-31 20:00:00 |
+------------------------------------------------------------+
1 row in set (0.00 sec)

Does CONVERT_TZ take daylight saving into account? [duplicate]

I am trying to set up a database that stores daily alert times as specified by users. For example, the user wants to receive an alert if some criterion is met each day between 7:00 AM and 7:30 AM. In trying to implement this, I need to accommodate daylight saving time. Here's my attempted solution:
Store the users local time zone (in long form, e.g. "US/Eastern") information in one table (say userInfo), and the alarm times in another table (say userAlarms).
When querying the userAlarms table, convert UTC time into the users local time as specified by the tz column stored in the userInfo table via CONVERT_TZ(UTC_TIME(), 'UTC', userInfo.tz).
Question 1. From my understanding, specifying the time zone name (like US/Eastern) should take daylight saving time into account. For example, calling CONVERT_TZ('00:00:00', 'UTC', 'US/EASTERN') on January 1 should yield '19:00:00', but on July 1 the call should yield '20:00:00'. Am I correct?
Question 2. If Q1 is correct, do I need to constantly update MySQL's time zone table to keep the time zone UTC offsets up to date?
Question 3. The sample given in the MySQL documentation SELECT CONVERT_TZ('2004-01-01 12:00:00','GMT','MET') yields "NULL" when run on my server. Could this be caused by not having the time zone tables set-up?
How can I check this?
If this yields null then the TZ tables have not been set up:
SELECT CONVERT_TZ(now(),'US/Eastern','US/Central');
If you do not have the time zone tables set up you could update the hour
offset in the user table and then do:
select utc_timezone() - interval user_timezone_offset_in_hours hour
from userinfo a
where user_id = 999;
You'd still need a way to update the user's time zone however.
If you are writing this for a web application you can get the time zone via javascript, here's an article that describes how (haven't tried this but it looks like it'll work).
A bit of an explanation with respect to 'interval' above...
One of the more trick constructs in MySQL is the use of the INTERVAL
keyword, best shown by example the (numeric value can be an expression or the field value)
select now() today, now() - interval 1 day yesterday;
+---------------------+---------------------+
| today | yesterday |
+---------------------+---------------------+
| 2011-05-26 13:20:55 | 2011-05-25 13:20:55 |
+---------------------+---------------------+
You can add them and subtract them anyway you like, this is why I never
bother with the date/time add/subtract/convert functions
select now() a, now() - interval 1 day + interval 4 hour + interval 8 minute b;
+---------------------+---------------------+
| a | b |
+---------------------+---------------------+
| 2011-05-26 13:24:16 | 2011-05-25 17:32:16 |
+---------------------+---------------------+
You can use negative numbers (should be good for negative time zone offsets)
these are the same:
select now() - interval 1 month a, now() + interval -1 month b;
+---------------------+---------------------+
| a | b |
+---------------------+---------------------+
| 2011-04-26 13:38:05 | 2011-04-26 13:38:05 |
+---------------------+---------------------+
I found this thread helpful and decided to share the doc page for importing this information. I did exactly as instructed below in CentOS and in RHEL and it worked flawlessly. I am now able to use the CONVERT_TZ function with arguments like "GMT" and "US/Eastern".
From the MySQL documentation this is how to import the MySQL time zone table information:
http://dev.mysql.com/tech-resources/articles/4.1/time.html
For all users running MySQL 4.1.3 or later on a Unix-based system (recall this doesn't work on Windows systems yet):
Populate the time zone tables.
To do so, run the mysql_tzinfo_to_sql program, provided with the MySQL distribution. mysql_tzinfo_to_sql reads the operating system time zone files and generates SQL statements from them. The SQL statements are then processed by mysql, to load the time zone tables.
To run mysql_tzinfo_to_sql successfully, one needs to know where the server machine's operating system time zone files are stored; check for a directory with a name similar to /usr/share/zoneinfo. Pass the directory name on the command line to mysql_tzinfo_to_sql, and send the output into the mysql program. Here's an example.
shell> mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql
Note: The above command assumes that "mysql_tzinfo_to_sql" and "mysql" are in your path. If they're not, you'll need to supply the full path to both when running the command, or switch into the mysql bin folder and use a command like so:
shell> ./mysql_tzinfo_to_sql /usr/share/zoneinfo | ./mysql -u root mysql
Q1: Yes, the CONVERT_TZ takes daylight savings time into account for you. This information and the time that DST starts/ends for each time zone is stored in the time_zone_* tables.
Q2: Yes, as stated in the mysql docs, the time zone information changes per the politics of each area. You'll have to update the time_zone_* tables every time a change occurs. Sucks to be IT sometimes, this is one of them.
Q3: These are the 5 timezone tables, query them to see if they have anything in them:
select * from mysql.time_zone_transition_type;
select * from mysql.time_zone_transition;
select * from mysql.time_zone_name;
select * from mysql.time_zone_leap_second;
select * from mysql.time_zone;

MySQL CONVERT_TZ()

I am trying to set up a database that stores daily alert times as specified by users. For example, the user wants to receive an alert if some criterion is met each day between 7:00 AM and 7:30 AM. In trying to implement this, I need to accommodate daylight saving time. Here's my attempted solution:
Store the users local time zone (in long form, e.g. "US/Eastern") information in one table (say userInfo), and the alarm times in another table (say userAlarms).
When querying the userAlarms table, convert UTC time into the users local time as specified by the tz column stored in the userInfo table via CONVERT_TZ(UTC_TIME(), 'UTC', userInfo.tz).
Question 1. From my understanding, specifying the time zone name (like US/Eastern) should take daylight saving time into account. For example, calling CONVERT_TZ('00:00:00', 'UTC', 'US/EASTERN') on January 1 should yield '19:00:00', but on July 1 the call should yield '20:00:00'. Am I correct?
Question 2. If Q1 is correct, do I need to constantly update MySQL's time zone table to keep the time zone UTC offsets up to date?
Question 3. The sample given in the MySQL documentation SELECT CONVERT_TZ('2004-01-01 12:00:00','GMT','MET') yields "NULL" when run on my server. Could this be caused by not having the time zone tables set-up?
How can I check this?
If this yields null then the TZ tables have not been set up:
SELECT CONVERT_TZ(now(),'US/Eastern','US/Central');
If you do not have the time zone tables set up you could update the hour
offset in the user table and then do:
select utc_timezone() - interval user_timezone_offset_in_hours hour
from userinfo a
where user_id = 999;
You'd still need a way to update the user's time zone however.
If you are writing this for a web application you can get the time zone via javascript, here's an article that describes how (haven't tried this but it looks like it'll work).
A bit of an explanation with respect to 'interval' above...
One of the more trick constructs in MySQL is the use of the INTERVAL
keyword, best shown by example the (numeric value can be an expression or the field value)
select now() today, now() - interval 1 day yesterday;
+---------------------+---------------------+
| today | yesterday |
+---------------------+---------------------+
| 2011-05-26 13:20:55 | 2011-05-25 13:20:55 |
+---------------------+---------------------+
You can add them and subtract them anyway you like, this is why I never
bother with the date/time add/subtract/convert functions
select now() a, now() - interval 1 day + interval 4 hour + interval 8 minute b;
+---------------------+---------------------+
| a | b |
+---------------------+---------------------+
| 2011-05-26 13:24:16 | 2011-05-25 17:32:16 |
+---------------------+---------------------+
You can use negative numbers (should be good for negative time zone offsets)
these are the same:
select now() - interval 1 month a, now() + interval -1 month b;
+---------------------+---------------------+
| a | b |
+---------------------+---------------------+
| 2011-04-26 13:38:05 | 2011-04-26 13:38:05 |
+---------------------+---------------------+
I found this thread helpful and decided to share the doc page for importing this information. I did exactly as instructed below in CentOS and in RHEL and it worked flawlessly. I am now able to use the CONVERT_TZ function with arguments like "GMT" and "US/Eastern".
From the MySQL documentation this is how to import the MySQL time zone table information:
http://dev.mysql.com/tech-resources/articles/4.1/time.html
For all users running MySQL 4.1.3 or later on a Unix-based system (recall this doesn't work on Windows systems yet):
Populate the time zone tables.
To do so, run the mysql_tzinfo_to_sql program, provided with the MySQL distribution. mysql_tzinfo_to_sql reads the operating system time zone files and generates SQL statements from them. The SQL statements are then processed by mysql, to load the time zone tables.
To run mysql_tzinfo_to_sql successfully, one needs to know where the server machine's operating system time zone files are stored; check for a directory with a name similar to /usr/share/zoneinfo. Pass the directory name on the command line to mysql_tzinfo_to_sql, and send the output into the mysql program. Here's an example.
shell> mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql
Note: The above command assumes that "mysql_tzinfo_to_sql" and "mysql" are in your path. If they're not, you'll need to supply the full path to both when running the command, or switch into the mysql bin folder and use a command like so:
shell> ./mysql_tzinfo_to_sql /usr/share/zoneinfo | ./mysql -u root mysql
Q1: Yes, the CONVERT_TZ takes daylight savings time into account for you. This information and the time that DST starts/ends for each time zone is stored in the time_zone_* tables.
Q2: Yes, as stated in the mysql docs, the time zone information changes per the politics of each area. You'll have to update the time_zone_* tables every time a change occurs. Sucks to be IT sometimes, this is one of them.
Q3: These are the 5 timezone tables, query them to see if they have anything in them:
select * from mysql.time_zone_transition_type;
select * from mysql.time_zone_transition;
select * from mysql.time_zone_name;
select * from mysql.time_zone_leap_second;
select * from mysql.time_zone;