I'm looking into ways of changing the time zone of my databases as my userbase is UK based, but my domain is US based. I've researched a number of options including php, PEAR time class, and MySQL functions. To me the simplest (largly because it starts and ends with MySQL) involves changing MySQL's time_zone parameter.
In preparation to do this I've just tried MySQL's ##global.time_zone call (like I'd better understand this before I start playing with it). It's produced a whole load of parameters that I didn't expect, and MySQL's site (link text) fails to mention - it's obviously an array, it returns lots of recognisable parameters like log-in protocols etc but I can't see any obvious parameters that return anything to do with timezones.
Do you have any experience of this? Can you advise?
Many thanks in advance.
There are 2 keys to dealing with timezones:
Store all of your date/times in the same timezone (can be any timezone, but using UTC does simplify things.)
Decide where in your application date/times will be converted to/from the storage timezone to the user display timezone. This can be done in the database abstraction layer, the application code, or in the page templates (any part of the MVC stack basically). The important thing is to be consistent.
Also remember that php has its own timezone setting independent of MySQL. If you're running php >5.1, you can change the php timezone at runtime using date_set_default_timezone().
If you decide to do the timezone conversion when you retrieve/store information in the database, MySQL's convert_tz() function will become your new best friend.
For me it's has been easiest to do the timezone conversions as part of the page templates. Date/times get stored in the database in the UTC timezone, and all application logic is in UTC as well. When outputting a date in HTML, I use a wrapper function for php's date() to output the date in the user's timezone:
function getTimezoneOffset($time, $timezone)
{
$t = new DateTime(date('Y-m-d H:i:s', $time));
$tz = new DateTimeZone($timezone);
$t->setTimeZone($tz);
return $t->getOffset();
}
function myDate($timezone, $format, $timestamp=false)
{
if (!$timestamp) $timestamp = time();
return date($format, $timestamp+getTimezoneOffset($timestamp, $timezone));
}
Likewise, when taking input from the user via forms, do the timezone conversion first so that your application has UTC date/times to work with.
Another option is to use PostgreSQL -- unlike MySQL it has timezone support for datetimes built-in.
The standard practice is to store any date/times you want to display in UTC (i.e. GMT) and either automatically fetch the timezone from the user's client using javascript, or asking the user where he lives.
Personally, I don't endorse the automatic method, since many users don't have the correct timezone set. But it can be useful to suggest a default to them.
Either way, once you have the user's timezone, just add it to the timestamps stored in UTC in the database before you display it.
Related
This question is specifically about future dates and times (for past values UTC is undoubtedly the first choice).
I'd wonder if anybody had a suggestion as to the "best" way to save a future date and time in a MySQL database (or for that matter also generally), particularly in a context where the column can hold times from different timezones. Considering that timezone rules may change UTC might not be the best option.
The most feasible option I found so far would be the save it as text in the location's local time, together with the location (eg. "America/*"). Unfortunately this solution might be more prone to data corruption and is definitely less convenient for calculations.
Could anybody think of something better?
Thanks in advance
First, I've written about this in extensive detail before, so please read my answers here and here, as well as this blog post by Lau Taarnskov.
With specific regard to MySQL, you generally don't want to use a TIMESTAMP field for the local time of a future event, as it will convert from the session's time zone to UTC at write time, and convert back from UTC to the session's time zone at read time. Even if these are the same time zone ids (which they don't have to be), there's no guarantee that the time zone data won't change for one or both of the time zones between when you write the data and when the event takes place.
Instead, use a DATETIME field, which does no implicit time zone conversions. You get the exact value out that you wrote. Store the local time of the event, and store a VARCHAR field containing the time zone identifier for the event. This is the only way to retain the user's original intent.
Rationale and edge cases are all described in the answers I gave previously.
The considerations for saving future datestamps are pretty much the same as for past datestamps.
(I call them datestamps because both DATETIME and TIMESTAMP are reserved words in MySQL. For the sake of discussion I want a word that doesn't imply either data type.)
If you're building a system to be used by people in multiple time zones, it's a good idea to ask each user for her time zone preference, and store it in her user profile. Then, when she logs in you can retrieve it, then do
SET time_zone = 'America/Halifax'
or whatever, naming the user's time zone preference.
If your MySQL server is running on a Linux, BSD, or other *nix system, these time zones come from the zoneinfo subsystem on that machine. zoneinfo gets updated when various national jurisdictions change time zone rules. The people who maintain popular distros routinely push updates to zoneinfo, so you'll be reasonably up to date. (If your MySQL server is running on a Windows host, do some reading about MySQL time zone stuff on that OS. It's more of a hassle to keep things up to date.)
Then, if you use TIMESTAMP data types for your datestamps, any time you retrieve a value, it is automatically translated from UTC to the local timezone before display. Any time you store a value it is automatically translated to UTC. The NOW() value is timestamp-like in this respect. So if you, for example, do
UPDATE appointment
SET datestamp = NOW() + INTERVAL 7 DAY
WHERE id = something
you'll store a UTC time that's a week after this moment. Then if you do
SELECT datestamp
FROM appointment
WHERE id = something
the user will see the time in her local timezone as set with SET timezone.
If you use DATETIME data types for your datestamps, you can offset them yourself when you store and retrieve them. When you store them, offset them from the local timezone to UTC. When you retrieve them, go the other way. Use CONVERT_TZ() for that.
UPDATE appointment
SET datestamp = CONVERT_TZ(NOW(), 'America/Halifax', 'UTC') + INTERVAL 7 DAY
WHERE id = something
SELECT CONVERT_TZ(datestamp, 'UTC', 'America/Halifax') datestamp
FROM appointment
WHERE id = something
Obviously, substitute your user's choice of timezone for 'America/Halifax' in these queries.
IF YOU POSSIBLY CAN AVOID IT don't store your datestamps with reference to a local time that changes from daylight savings to standard time. If you do that, you will have ongoing glitches on the changeover days for the lifetime of your application. I know this because I've inherited a couple of systems that worked that way. Bad idea. UTC: good idea.
Here's the scenario. I'm using a MySQL/NodeJS/Sequelize stack on the server, and I have a request I want to perform.
There are anywhere from 1000-2000 entries that are retrieved from the request, but I don't need the full list of entries. I want the summary of the entries after they have been grouped by the day the entry was made, which condenses down to about 5-10 objects in an array.
If I group by day on the server, it may group them differently from the time zone of the client. And if I send it to the client, then I have to send 1-2k entries for analysis, but it will group them correctly.
How would you handle this scenario?
Use UTC timestamps for everything. For example open the Chrome console and enter this:
// Milliseconds since 1/1/1970
new Date().getTime()
1438263135084
// Human friendly without timezone
new Date().toISOString()
"2015-07-30T13:32:15.715Z"
// Human friendly with timezone
new Date().toString()
"Thu Jul 30 2015 09:32:21 GMT-0400 (EDT)"
All of these are the same (time since 1/1/1970), but simply formatted differently. Using either of the first two will allow you to send dates that can be interpreted and formatted correctly regardless of timezone.
Interesting reading: https://en.wikipedia.org/wiki/Coordinated_Universal_Time
Perhaps you could convert the dates to the user's timezone Django MySQL group by day with timezone - but that of course requires you to know the user's timezone, and caching will be harder
In sequelize, it would be expressed something like
sequelize.fn('CONVERT_TZ', sequelize.col('date'), 'UTC', user_tz)
First off, thanks all for your suggestions on how to handle this. Ultimately, I resolved this issue by doing the following.
First, in the request from the client, I used the moment js method .utcOffset() to get the offset value.
Then, once I had the offset value, I included it into the API query as ?offset=(value here)
Finally, on the server side, I used this value to interpret the local timezone offset of the client, and then grouped all the data there, and send the formatted data back to the client. It resulted in a much faster, much, much smaller response query.
It was a fairly simple solution - I'm bugged it took me to long to come up with it.
In my DB I am storing the users last login date/time using NOW(). I realize this stores using my servers timezone which I have no problem with. While I plan to use this for my own purposes (they say they never logged in or yada yada) I also want to be able to display this last login time on their profile for them to see as well.
What would be the best practice for this? At the moment I am thinking I should just let them choose/edit the timezone from their profile and then this in turn will convert the NOW() value stored in the DB to their time. This would require another column in the DB, but would allow me to do the proper calculation for their time as well as list the current timezone they have selected for their profile.
For instance, I'm in est so I could show 2013-09-04 02:46:05 EST to them on their account profile, but also let them edit their timezone to correct it for their display while keeping the original est date in the DB (my servers config tz).
I realize the giving the user their last login date/time seems kind of useless, but for this website it actually will have some value to them so I want to make sure it reflects 'their' time and makes sense to them.
Is this the best practice or better ideas out there?
You cannot find out the timezone of the user from the server. Period.
I strongly recommend you store the login time using UTC (from getutcdate()) as it is unambiguous. If you use local time then daylight savings transitions result in "repeated hours" where the same time will be recorded for events which occur an hour apart. This makes it impossible to work out what time the event actually occurred.
If you don't know the timezone of the user, just display UTC and tell the user it is UTC so they are not confused.
However if you want to know the timezone of the user, you must get the client software to tell you, perhaps by asking the user, and storing this in his preferences. Alternatively you may be able to guess from JavaScript.
See also this question, which discusses the timezone issue in more detail:
Storing DateTime (UTC) vs. storing DateTimeOffset
We are developing an application in C# 4 that uses SQL Server 2008 R2 as backend. SQL Server Compact 4 is also used for disconnected clients in a very few rare scenarios. We are wondering what's the best way to store date/time data into these databases so that:
Data containing different time offsets (coming from different time zones) can co-exist nicely. This means being sorted and compared.
Data in SQL Server 2008 R2 and SQL Server Compact 4 can be transferred back and forth seamlessly; this is a secondary requirement that should not compromise the chosen design.
Our main concern is to preserve the local time for each recorded event, but without losing the ability to compare and sort events that have been generated from different time zones and therefore have different time offsets.
We have considered the datetimeoffset data type, since it stores the time offset and because it maps nicely to .NET's DateTimeOffset. However, it is not supported in SQL Server Compact 4. An alternative would be to remove offset info from the database and use a simple datetime data type, so that every piece of data in the database is normalized, and the issues with Compact are fewer. However, this introduces the problem that offset info would need to be reconstructed somehow on retrieval before the user sees the data.
So my question is, are there any best practices or guidelines on how to store date/time values in SQL Server, taking into account that we will need to deal with different time zones, and making the interoperability between 2008 R2 and Compact 4 as easy as possible?
Thank you.
It sounds like the relevant points are:
Your incoming data is a good fit for DateTimeOffset
You only care about the offset at that particular time, so you don't need a real time zone. (An offset isn't a time zone.)
You do care about that original offset - you can't just normalize everything to UTC and ignore the offset entirely.
You want to query on the local time.
It does sound like DateTimeOffset is basically the most appropriate type in this case. You should make sure everyone on the team is clear about what it means though - the offset is the offset when the data was originally received. If you want to display that instant in time in a different time zone, you effectively need to go back to UTC, and find out what the offset would be in that display time zone. It's easy to get confused about this sort of thing :)
If you need to maintain the data in SqlServerCE with full fidelity, you'll probably want a DateTime field and then a separate field for the offset (e.g. in minutes, or as a TimeSpan if SqlServerCE supports that).
You are probably right to use DateTimeOffset on the server. You might also want to read my answer on DateTime vs DateTimeOffset.
On the client, where you are using SQLCE, store a DateTime with UTC values. When you send the data to the server, you can use the local time zone of the client to determine the DateTimeOffset that the UTC value corresponds to.
If it's possible that the user might be changing their time zones, then you might also need to store the time zone's id in the client database. But you would just use this during conversion. There's no need to send it to the server unless you might be editing those values on the server or in some other client.
Don't try storing the time on the client in the local time of the client. You will encounter abiguities. For example, when daylight saving time rolls backwards, you don't want two different possible UTC times for the same local time.
Why not use datetime and always store the value as a UTC value, then you can format it to the end users (display) time zone when required.
What's the best way to store timezone information with dates/times in a uniform way so people can enter times from anywhere in the world in their own local time? Other users would then have the ability to view the times in their own local time and the original localtime on the web page.
Store the time data as GMT. Display it using local time. This requires a simple offset from GMT. Now if the politicians would leave the starting and end date for Day light savings time alone...
I'd agree with Staale, but add that times should be stored in Zulu time (commonly called UTC), to make for easier translation into other timezones. Don't rely on storing data in the timezone used by your server.
I would suggest inserting the date in UTC time zone. This will save you a lot of headache in the future (Daylight saving problems etc...)
"INSERT INTO abc_table (registrationtime) VALUES (UTC_TIMESTAMP())"
When I query my data I use the following PHP script
<? while($row = mysql_fetch_array($registration)){
$dt_obj = new DateTime($row['message_sent_timestamp']." UTC");
$dt_obj->setTimezone(new DateTimeZone('Europe/Istanbul'));
echo $formatted_date_long=date_format($dt_obj, 'Y-m-d H:i:s'); } ?>
You can replace the datetimezone value with one of the available php timezones here:
My view has always been to store exact numeric timestamps (seconds or milliseconds since midnight 1st january 1970 GMT), as that's a format that is easily converted to an actual date and time in any timezone.
The downside of this is that the data isn't as immediately viewable through normal SQL tools. The mysql cli client does have methods for this though (FROM_UNIXTIMESTAMP).
Always store as times as UTC/GMT. Check the mysql docs to see what column type is best for the range of dates and/or precision that you want to support.
On input, convert to UTC and store the client timezone offset in a second column. Then you can easily convert to any timezone you want for display, or use the submitted offset to recreate the original timestamp.