We've been working on implementing timezone support for our Web app.
This great SO post has helped us a bunch: Daylight saving time and time zone best practices
We've implelmented the OLSON TZ database in MYSQL and are using that for TZ conversions.
We're building a scheduling app so:
We are storing all our bookings which occur on a specific date at a specific time in UTC time in DateTime fields and converting them using CONVERT_TZ(). This is working great.
What we aren't so sure about is stuff like vacations and breaks:
Vacations are just Date references and don't include a time portion. Because CONVERT_TZ() doesn't work on date objects we are guessing that we are best to just store the date value as per the user's timezone?
id1 id3 startDate endDate
-----------------------------
3 6 2010-12-25 2011-01-03
4 3 2010-09-22 2010-09-26
Same thing with recurring breaks during stored for each day of the week. We currently store their breaks indexed 0-6 for each day of the week. Because these are just time objects we can't use CONVERT_TZ() and assume we should just store them as time values in the user's time zone?
bID sID dayID startTime endTime
--------------------------------
1 4 1 12:00:00 14:00:00
2 4 4 13:30:00 13:30:00
In this case with vacations and breaks we would only compare them to booking times AFTER the booking times have been converted to the user's local time.
Is this the correct way to handle things, or should we be storing both vacations and breaks in some other way so that we can convert them to UTC (not sure how this would work for breaks).
Thanks for your assistance!
The two storage formats look fine. You just need to convert them to the user's local time when you pull them out of the table.
Actually, for the breaks table I presume they're already nominally in local time, so you just compare directly against the local time of the appointment.
I don't understand your question well enough to say my answer is 100% correct for you. But I think what you need to do is store the DateTime in "local" time and also store the timezone. This way you have it correct even if daylight savings time shifts (which happens).
Good article at http://blogs.windwardreports.com/davidt/2009/11/what-every-developer-should-know-about-time.html (yes by me).
Related
In database, the table-> date column showing the correct date and time.
in database table column :
2020-08-25 04:00:32.217609
But when I am fetching the same date, it's showing the exactly 24 hrs old date and time.
fetched from database :
2020-08-24T16:00:32.217Z
I think it's about local timezone and also the format is different when fetching. I am trying to understand the issue and then looking for solution.
Note: I am fetching the data using typeorm queryBuilder.
Yes, they are the same time in different time zones. The first is in your local time zone (New Zealand Standard Time) 12 hours ahead of UTC. The Z at the end of the second indicates it is in UTC, 12 hours behind you.
The other difference is in the fractional seconds. Your database is storing in microseconds. Your program is storing in milliseconds, or only displaying milliseconds.
For instance, I have a blog where users can comment and I want everyone can see how long ago the comment was posted, for example: 5 minutes ago OR 3 hours ago.
So if a guy in London posts a comment and a guy in India visits the page, they both should see "1 minute ago" and on hover should see the time relative to their timezone. (10pm in London, 3.30am in India).
My current solution in mind is to use varchar(25) data type and store the time as ISO-8601 (e.g. 2019-12-12T21:46:42+00:00)
Using this I can get the timezone of the commenter and convert the time to the current user's timezone. It works perfectly.
But I wonder if there is a better / more elegant way to do it?
So far I tried using DATETIME and TIMESTAMP data types but they do not seem to be useful in this scenario. I read online that TIMESTAMP is supposed to store time in UTC timezone and send it back in user's timezone but that did not happen for me, it got saved in my local time instead. And yes, I did not specify any time while saving data, MySQL used the CURRENT_TIMESTAMP.
Any thoughts or ideas?
I'd recommend storing all your dates/times in one universal format in your database and UTC would be the best candidate for this.
That way, regardless of their location, it's easy for you to say 1 minute ago...
If you need to display the full date/time on the front-end, you'd need to convert the time from UTC to that user's location, which you can do via PHP's handy DateTime functions:
https://www.php.net/manual/en/datetime.settimezone.php
Which type I should use to store current date + time in UTC?
Then to be able to convert UTC date to specific timezone?
Now I use TIMESTAMP type and CURRENT_TIMESTAMP.
It stores data like: 2019-08-19 20:44:11
But minutes are different that real UTC time, I dont know why.
My server time is local. It is correct under Windows Server
It is up to you to decide the best way to solve timezone problem when users and server has different locale.
No matter the case and the app (mobile, web, etc.) the problem is the same. You should find the best and easiest in your case way to handle time zones.
Here are few options that you can use:
MySQL
From MySQL Date and Time Types - you can create table fields that will hold your date and time values.
"The date and time types for representing temporal values are DATE, TIME, DATETIME, TIMESTAMP, and YEAR. Each temporal type has a range of valid values, as well as a “zero” value that may be used when you specify an invalid value that MySQL cannot represent. The TIMESTAMP type has special automatic updating behavior, described later."
In respect to MySQL Data Type Storage Requirements read the link and make sure you satisfy the table storage engine and type requirements in your project.
Setting the timezone in MySQL by:
SET time_zone = '+8:00'
To me this is a bit more work to handle, but the data is fully loaded, managed and updated by MySQL. No PHP here!
Using MySQL might seem like a better idea (that's what I'd like to think), but there's a lot more to it.
To be able to choose, you will have to make an educated decision. There's a lot to cover in regards to using MySQL. Here's a practical article that goes into the rabbit hole of using MySQL to manage date, time and timezone.
Since you didn't specify how you interface the database, here's a PHP example and functions to handle the date, time and time zones.
PHP
1. Save date, time and time zone
E.g. Chicago (USA - Illinois) - UTC Offset UTC -5 hours
You can save the date time
2015-11-01 00:00:00
and the time zone
America/Chicago
You will have to work out DST transitions and months having different numbers of days.
Here's a reference to the DateTime to work out any timezone and DST differences:
DateTime Aritmetic
2. Unix Timestamp and Time Zone
Before we go into the details of this option we should be aware of the following:
The unix time stamp is a way to track time as a running total of seconds. This count starts at the Unix Epoch on January 1st, 1970 at UTC. Therefore, the unix time stamp is merely the number of seconds between a particular date and the Unix Epoch. It should also be pointed out (thanks to the comments from visitors to this site) that this point in time technically does not change no matter where you are located on the globe. This is very useful to computer systems for tracking and sorting dated information in dynamic and distributed applications both online and client side.
What happens on January 19, 2038?
On this date the Unix Time Stamp will cease to work due to a 32-bit overflow. Before this moment millions of applications will need to either adopt a new convention for time stamps or be migrated to 64-bit systems which will buy the time stamp a "bit" more time.
Here's how the timestamp works:
08/19/2019 # 8:59pm (UTC) translates to 1566248380 seconds since Jan 01 1970. (UTC)
Using the PHP date() function you can format to anything you want like:
echo date('l jS \of F Y h:i:s A', 1566248380);
Monday 19th of August 2019 08:59:40 PM
or MySQL:
SELECT from_unixtime(2147483647);
+--------------------------------------+
| from_unixtime(2147483647) |
+--------------------------------------+
| 2038-01-19 03:14:07 |
+--------------------------------------+
More example formats that you can convert to:
08/19/2019 # 8:59pm (UTC)
2019-08-19T20:59:40+00:00 in ISO 8601
Mon, 19 Aug 2019 20:59:40 +0000 in RFC 822, 1036, 1123, 2822
Monday, 19-Aug-19 20:59:40 UTC in RFC 2822
2019-08-19T20:59:40+00:00 in RFC 3339
The PHP Date() function can be used as a reference.
Again you will have to save the time zone:
America/Chicago
Set the PHP script time zone for your users by using date_default_timezone_set() function:
// set the default timezone to use. Available since PHP 5.1
date_default_timezone_set('UTC');
date_default_timezone_set('America/Chicago');
You can't store a date/time with time zone information.
MySQL does not store the time zone information on either DATETIME or TIMESTAMP. They are assumed to be on the server time zone.
The only ugly work around is to set the whole MySQL server/vm/docker container to UTC.
This question is specifically about future dates and times (for past values UTC is undoubtedly the first choice).
I'd wonder if anybody had a suggestion as to the "best" way to save a future date and time in a MySQL database (or for that matter also generally), particularly in a context where the column can hold times from different timezones. Considering that timezone rules may change UTC might not be the best option.
The most feasible option I found so far would be the save it as text in the location's local time, together with the location (eg. "America/*"). Unfortunately this solution might be more prone to data corruption and is definitely less convenient for calculations.
Could anybody think of something better?
Thanks in advance
First, I've written about this in extensive detail before, so please read my answers here and here, as well as this blog post by Lau Taarnskov.
With specific regard to MySQL, you generally don't want to use a TIMESTAMP field for the local time of a future event, as it will convert from the session's time zone to UTC at write time, and convert back from UTC to the session's time zone at read time. Even if these are the same time zone ids (which they don't have to be), there's no guarantee that the time zone data won't change for one or both of the time zones between when you write the data and when the event takes place.
Instead, use a DATETIME field, which does no implicit time zone conversions. You get the exact value out that you wrote. Store the local time of the event, and store a VARCHAR field containing the time zone identifier for the event. This is the only way to retain the user's original intent.
Rationale and edge cases are all described in the answers I gave previously.
The considerations for saving future datestamps are pretty much the same as for past datestamps.
(I call them datestamps because both DATETIME and TIMESTAMP are reserved words in MySQL. For the sake of discussion I want a word that doesn't imply either data type.)
If you're building a system to be used by people in multiple time zones, it's a good idea to ask each user for her time zone preference, and store it in her user profile. Then, when she logs in you can retrieve it, then do
SET time_zone = 'America/Halifax'
or whatever, naming the user's time zone preference.
If your MySQL server is running on a Linux, BSD, or other *nix system, these time zones come from the zoneinfo subsystem on that machine. zoneinfo gets updated when various national jurisdictions change time zone rules. The people who maintain popular distros routinely push updates to zoneinfo, so you'll be reasonably up to date. (If your MySQL server is running on a Windows host, do some reading about MySQL time zone stuff on that OS. It's more of a hassle to keep things up to date.)
Then, if you use TIMESTAMP data types for your datestamps, any time you retrieve a value, it is automatically translated from UTC to the local timezone before display. Any time you store a value it is automatically translated to UTC. The NOW() value is timestamp-like in this respect. So if you, for example, do
UPDATE appointment
SET datestamp = NOW() + INTERVAL 7 DAY
WHERE id = something
you'll store a UTC time that's a week after this moment. Then if you do
SELECT datestamp
FROM appointment
WHERE id = something
the user will see the time in her local timezone as set with SET timezone.
If you use DATETIME data types for your datestamps, you can offset them yourself when you store and retrieve them. When you store them, offset them from the local timezone to UTC. When you retrieve them, go the other way. Use CONVERT_TZ() for that.
UPDATE appointment
SET datestamp = CONVERT_TZ(NOW(), 'America/Halifax', 'UTC') + INTERVAL 7 DAY
WHERE id = something
SELECT CONVERT_TZ(datestamp, 'UTC', 'America/Halifax') datestamp
FROM appointment
WHERE id = something
Obviously, substitute your user's choice of timezone for 'America/Halifax' in these queries.
IF YOU POSSIBLY CAN AVOID IT don't store your datestamps with reference to a local time that changes from daylight savings to standard time. If you do that, you will have ongoing glitches on the changeover days for the lifetime of your application. I know this because I've inherited a couple of systems that worked that way. Bad idea. UTC: good idea.
I'm running a matlab function (fastinsert) to insert data into MySQL. The results are correct for the whole year except for 1 hour in March, during daylight saving. In fact it seems that I cannot insert data between 2:00am and 3:00am on that day.
For example with:
ts = 2006 3 26 2 30 0
looking within the matlab function I found that the problem lies into:
java.sql.Timestamp(ts(1)-1900,ts(2)-1,ts(3),ts(4),ts(5),secs,nanosecs)
that gives as a result:
2006-03-26 03:30:00.0
How can I solve this?
I've run into similar problems in storing datetime on many occasions. Treating the value as a derived value seems to make the most sense. In other words, instead of storing the local time store the value as GMT and Time Zone. Then derive the appropriate value when you query the data.
This has the added benefit of making it possible to store values from multiple locations without having to worry about confusion down the road.