I have table with dates saved in UTC +00:00. Pre-mysql call PHP sets timezone offset with or without daylight saving:
date_default_timezone_set($tzone);
$tzoffset = date("P");
Mysql call adds hours depending of users timezone:
CONVERT_TZ(d.dateTime,'+00:00', '".$tzoffset."') AS dateTime,
Yesterdays 09:00 before daylight saving, today is displayed as 10:00. How to retrieve date from before daylight saving at it's original time? Some sort of IF logic in Mysql call as I use in PHP?:
$date = new DateTime( $row[ 'dateTime' ] );
$tzDaylightSaving = $date->format("I"); //checking if daylight saving for specific time is active
if ($tzDaylightSaving == 1)
{
$date->add(new DateInterval('PT1H')); //if daylight saving is active, add one hour
}
This isn't really a MySQL problem. In the call to CONVERT_TZ, as far as MySQL sees it, the client SQL is specifying a fixed offset from UTC. From the code, it looks like that that offset is derived from current date and time, independent of whether the datetime value stored in dateTime column.
MySQL is perfectly capable of handling timezone that observe daylight saving time. We could pass in the actual time_zone value (assuming the mysql.time_zone% tables are populated) e.g. PST8PDT, or America/Chicago
CONVERT_TZ( t.datetime_utc ,'+00:00','PST8PDT')
CONVERT_TZ( t.datetime_utc ,'+00:00','America/Chicago')
Or, we could just return the UTC value from MySQL, and do the conversion in the code.
If we are committed to sending a fixed offset in the SQL text, and returning a datetime value offset from UTC, then in the code, we need to account for that.
As the code stands, the value being returned by MySQL is not in time_zone $tzone. It's a value with a wonky fixed offset from UTC. When we go to instantiate that value as a PHP DateTime, the code is using the wrong time_zone. We would need to interpret that MYSQL value as being a fixed offset from UTC.
At the point in the code it's doing the if check, it's too late. The $date value is wrong because it's being interpreted as if its in the default time zone $tzone... and it's not. The value from MySQL has been shifted a fixed number of hours.
Related
I am using spring boot and mysql.
At first, one of my time field using Date type and I gotta use the #JsonFormat(shape=JsonFormat.Shape.STRING, pattern=Constant.DATE_TIME_FORMAT, timezone = "....") to take care of the time conversion due to different timezone. And then I changed it from Date to LocalDateTime to ensure there's no time conversion and remove the timezone property from above annotation.
But I was wrong, in UAT enrivonment, the input data saved in UI(swagger) changed '12:00:00' to '04:30:00' in database.
I then tried to test by adjusting time zone in my machine and also the current time zone in mysql server.
Here's the result after several attemps:
Back-end Timezone, Database Timezone | Result
BE: +8:00, DB: +5:00 | 12h -> 9h
BE: +0:00, DB: +5:00 | 12h -> 17h
BE: +8:00, DB: +0:00 | 12h -> 4h
The data type for table column is time because I just want to save and display time so it doesn't matter which date so I'm using 1970-01-01 as default date.
The problem is I dont want any of time conversion due to the difference between timezone for BE and DB, I want the data saved in database to be exact as the input time on UI (UI: 12h -> DB: 12h).
This is my field in entity:
#ApiModelProperty(example = "1970-01-01 12:00:00")
#JsonFormat(shape=JsonFormat.Shape.STRING, pattern=Constant.DATE_TIME_FORMAT)
private LocalDateTime monStartTime;
Question: is there any way that can avoid time conversion despite the difference in timezone between BE and DB?
You can take String as datatype in Backend.
private Sting monStartTime;
No need to change in Database. Database column should be time as you already have that.
Try this. It will not convert your timezone.
There's lots of stuff on the internet about converting MySQL timestamps, how it works etc. But what I need is the opposite: knowing that MySQL stores every datetime data as UTC in TIMESTAMP fields, all I want is to direclty retrieve the stored UTC value without MySQL messing around the datetime with system/server/connection timezones.
You see, we've built a simple node.js feeder which reads from several third-part MySQL databases (so I can't change their timezone settings) and save the gathered data to a Elasticsearch, as a "denormalization process". As the original data comes from different timezones, I need to store them in UTC, so I can easily coordinate further GETs and aggregations.
I know I can set the connection timezone on the fly and I know I can change every timestamp field fetched in my node application, but since MySQL engine already stores timestamps in UTC, why should I add any other step if I could simply get it directly, without converting functions or costly data processings?
In a nutshell, I'd like to know: is there a way to bypass MySQL automatic GMT conversion?
MySQL provides a UNIX_TIMESTAMP function which returns a raw integer value. And that isn't subject to timezone conversions at all.
SELECT UNIX_TIMESTAMP( timestamp_col ) ...
But that returns a raw integer, not a datetime. The client would need to do the conversion into a "datetime" type object, if that's needed.
Another option would be to use the CONVERT_TZ function to convert to UTC from the session time_zone setting.
SELECT CONVERT_TZ( timestamp_col, ##session.time_zone,'+00:00')
But, that doesn't really "bypass" timezone conversion. One downside of this approach is if the session time_zone is affected by daylight saving time changes, there's ambiguity with a one hour period each year when the clock "falls back" one hour. e.g. Sunday, Nov 1 2015 2AM CDT transition to Sunday Nov 1 2015 1AM CST. (Converting back from UTC, if we get 1:30 AM in the session time_zone, we don't "know" if that's CDT or CST. And the conversion back to UTC doesn't know which it is either.)
Another option (which I think you already mentioned) is changing the session time_zone to UTC. Then you could just return the timestamp_col value as UTC. You could save the current time_zone setting, and set it back when you are done, e.g.
SET #save_session_time_zone := ##session.time_zone ;
SET time_zone = '+00:00' ;
SELECT timestamp_col ...
SET time_zone = #save_session_time_zone ;
But your client Connector might do some not-so-helpful conversions when the time_zone of the MySQL database session doesn't match the time_zone of the client, like the funky shenanigans the JDBC driver (MySQL Connector/J) does. (That concern isn't limited to returning UTC; that's a concern whenever the time_zone of the client doesn't match the time_zone of the database session.)
It looks like there's no way to get the original UTC value from a MySQL field; every single function uses the timezone setting, be that SYSTEM or any other you configure.
The way MySQL forces you to use a date conversion is, at least, very constraining. For example, say you have a MySQL server set to a timezone with GMT -03:00 and GMT/DST -02:00 and you store a datetime like '2016-07-01 10:00:00'. If you select this value after the DST has ended, you'll get '2016-07-01 09:00:00'.
You can't tell what time it is for sure unless you store the GMT offset separately or you previously know what timezone the server was when it was stored.
We used the second approach. We saved the server timezone and used it to calculate the offset and return an ISO datetime, so future calculations can be made easily.
DROP FUNCTION IF EXISTS `iso_datetime`;;
CREATE FUNCTION `iso_datetime` (
p_datetime TIMESTAMP
) RETURNS VARCHAR(25)
READS SQL DATA
BEGIN
DECLARE _timezone VARCHAR(255) DEFAULT NULL;
DECLARE _offset VARCHAR(6) DEFAULT NULL;
SET _timezone = (SELECT timezone FROM network);
SET _offset = (SELECT SUBSTRING(TIMEDIFF(p_datetime,CONVERT_TZ(p_datetime, _timezone,'UTC')), 1,6));
RETURN CONCAT(DATE_FORMAT(p_datetime, '%Y-%m-%dT%H:%i:%S'), _offset);
END;
In order to do so, you have to load timezone info into MySQL, so the server can calculate the tz offset of the date for you.
My purpose is since the time I login my page, I want my web to show how many updated data in the database. My code is like this
$current = $_SESSION['date'];
$query2 = "SELECT * FROM gmaptracker1 WHERE datetime >= '$current'";
When I echo the $current, it showed 27/09/14 : 06:53:24, so the $current is correct, however, when I request the number of database where date>='$current', I get zero, although I have inserted to the database the data with datetime 28/09/14 : 06:53:24 and 29/09/14 : 06:53:24.
Can anyone help me to get out of this, please?
Few things,
It seems like your code is vulnerable to SQL Injection. Just because you retrieve the content of the date from a session, it doesn't mean that it's safe.
Also, why do you need it to be in a session variable? If you always want to retrieve dates bigger than NOW() you can just write your query this way:
SELECT * FROM gmaptracker1 WHERE datetime >= NOW()
The part that caught my attention was the format you're storing the dates.
You said that when you echo'ed $_SESSION['date'] the value was: 27/09/14 : 06:53:24
Now, that does not look like the date format at all. Is your column actually a datetime or timestampcolumn?
If it's a VARCHAR or any other type other than datetime or timestamp, then there's no way for MySQL to know that you're trying to retrieve dates that occur in the future.
If you already have data stored, then it isn't going to be as easy as changing the data type because you already have data, and your data is in the wrong format. The format that MySQL stores datetime information is as follows:
YYYY-MM-DD HH:MM:SS
Based on the comments you left, you don't need the time > NOW(), you need the time when you log in. Now it makes sense why you're storing that time in a variable.
The problem is the format you're storing it.
Since you're using PHP, then you have to store the time this way:
$time = new DateTime();
$_SESSION['date'] = $time->format("Y-m-d H:i:s");
I have a very simple code for a cron job that makes a date entry into an SQL DB:
$qry_cron_test = "INSERT INTO ".$tblprefix."cron_test SET
create_datetime = '".date("Y-d-m H:i:s")."'";
$rs_cron_test = $db -> Execute($qry_cron_test);
The problem is the following:
Between 1st and 12th of every month the date entry is like this - 2014-10-03 07:30:39, which is what i want.
However, when the current date is between 13th and the end of the month, the date entry looks like this - 0000-00-00 00:00:00. Then when 1st comes the entires are all ok again.
I tested this on couple of servers and also locally on Xampp always with the same result.
Any suggestions? What could be possibly wrong?
You have month and day the wrong way around.
$qry_cron_test = "INSERT INTO ".$tblprefix."cron_test SET
create_datetime = '".date("Y-m-d H:i:s")."'";
$rs_cron_test = $db -> Execute($qry_cron_test);
date("Y-m-d H:i:s")
I recommend that, unless you need milisecond information, you always store date information in Unix Timestamp. It is lighter to store, since it is only a integer value, is faster to retrieve and is universal, since it is always based on UTC.
Specially in PHP, converting date information to (time() and strtotime) and from (date()) a unix timestamp is pretty easy. This way no matter where your user is, you can always show correct information in local time with almost no effort.
Wouldn't it be simpler to just do this:
insert into cron_test
create_datetime
values
(current_timestamp)
I have a small Django project that imports data dumps from MongoDB into MySQL. Inside these Mongo dumps are dates stored in epoch time. I would expect epoch time to be the same regardless of time zone but what I am seeing is that the Django TIME_ZONE setting has an effect on the data created in MySQL.
I have been testing my database output with the MySQL UNIX_TIMESTAMP function. If I insert a date with the epoch of 1371131402880 (this includes milliseconds) I have my timezone set to 'America/New_York', UNIX_TIMESTAMP gives me 1371131402, which is the same epoch time excluding milliseconds. However if I set my timezone to 'America/Chicago' I get 1371127802.
This is my code to convert the epoch times into Python datetime objects,
from datetime import datetime
from django.utils.timezone import utc
secs = float(epochtime) / 1000.0
dt = datetime.fromtimestamp(secs)
I tried to fix the issue by putting an explict timezone on the datetime object,
# epoch time is in UTC by default
dt = dt.replace(tzinfo=utc)
PythonFiddle for the code
I've tested this Python code in isolation and it gives me the expected results. However it does not give the correct results after inserting these object into MySQL through a Django model DateTimeField field.
Here is my MySQL query,
SELECT id, `date`, UNIX_TIMESTAMP(`date`) FROM table
I test this by comparing the unix timestamp column in the result of this query against the MongoDB JSON dumps to see if the epoch matches.
What exactly is going on here? Why should timezone have any effect on epoch times?
Just for reference, I am using Django 1.5.1 and MySQL-python 1.2.4. I also have the Django USE_TZ flag set to true.
I am no python or Django guru, so perhaps someone can answer better than me. But I will take a guess at it anyway.
You said that you were storing it in a Django DateTimeField, which according to the documents you referenced, stores it as a Python datetime.
Looking at the docs for datetime, I think the key is understanding the difference between "naive" and "aware" values.
And then researching further, I came across this excellent reference. Be sure the read the second section, "Naive and aware datetime objects". That gives a bit of context to how much of this is being controlled by Django. Basically, by setting USE_TZ = true, you are asking Django to use aware datetimes instead of naive ones.
So then I looked back at you question. You said you were doing the following:
dt = datetime.fromtimestamp(secs)
dt = dt.replace(tzinfo=utc)
Looking at the fromtimestamp function documentation, I found this bit of text:
If optional argument tz is None or not specified, the timestamp is converted to the platform’s local date and time, and the returned datetime object is naive.
So I think you could do this:
dt = datetime.fromtimestamp(secs, tz=utc)
Then again, right below that function, the docs show utcfromtimestamp function, so maybe it should be:
dt = datetime.utcfromtimestamp(secs)
I don't know enough about python to know if these are equivalent or not, but you could try and see if either makes a difference.
Hopefully one of these will make a difference. If not, please let me know. I'm intimately familiar with date/time in JavaScript and in .Net, but I'm always interested in how these nuances play out differently in other platforms, such as Python.
Update
Regarding the MySQL portion of the question, take a look at this fiddle.
CREATE TABLE foo (`date` DATETIME);
INSERT INTO foo (`date`) VALUES (FROM_UNIXTIME(1371131402));
SET TIME_ZONE="+00:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
SET TIME_ZONE="+01:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
Results:
DATE UNIX_TIMESTAMP(`DATE`)
June, 13 2013 13:50:02+0000 1371131402
June, 13 2013 13:50:02+0000 1371127802
It would seem that the behavior of UNIX_TIMESTAMP function is indeed affected by the MySQL TIME_ZONE setting. That's not so surprising, since it's in the documentation. What's surprising is that the string output of the datetime has the same UTC value regardless of the setting.
Here's what I think is happening. In the docs for the UNIX_TIMESTAMP function, it says:
date may be a DATE string, a DATETIME string, a TIMESTAMP, or a number in the format YYMMDD or YYYYMMDD.
Note that it doesn't say that it can be a DATETIME - it says it can be a DATETIME string. So I think the actual value being implicitly converted to a string before being passed into the function.
So now look at this updated fiddle that converts explicitly.
SET TIME_ZONE="+00:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
SET TIME_ZONE="+01:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
Results:
DATE CONVERT(`DATE`, CHAR) UNIX_TIMESTAMP(CONVERT(`DATE`, CHAR))
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371131402
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371127802
You can see that when it converts to character data, it strips away the offset. So of course, it makes sense now that when UNIX_TIMESTAMP takes this value as input, it is assuming the local time zone setting and thus getting a different UTC timestamp.
Not sure if this will help you or not. You need to dig more into exactly how Django is calling MySQL for both the read and the write. Does it actually use the UNIX_TIMESTAMP function? Or was that just what you did in testing?