I am trying to insert a date value in MySql table name person and column name regdate with data type = datetime. I am setting a value e.g. '2019-08-21 20:25:20' but after saving +5:30 hours get added and value which gets stored is '2019-08-22 03:55:20'. Generating the date value using below Java code
Timestamp curDate = Timestamp.valueOf(Instant.now().atZone(ZoneId.of("Asia/Kolkata")).toLocalDateTime());
and then using .setTimestamp(1, curdate); in INSERT query.
I have checked that the timezone of MySql is set to IST (GMT+0530). App Server timezone is also set to IST. But I am not able to understand why +5:30 hours are getting added even if I explictly setting the date value.
I have tried setting timezone in connection string as ?serverTimezone=Asia/Kolkata but didn't work.
But if I run the same code using my local machine connecting same MySql instance, I get no problem and same value gets stored without addition of 5:30 hours. I checked App Server timezone and it is IST.
MySql version - 5.7.17-log
mysql-connector-java - 8.0.15
Am I missing something?
You have a few problems here.
Avoid legacy date-time classes
First of all, you are mixing the terrible legacy date-time classes (java.sql.Timestamp) with the modern java.time classes. Don’t. Use only classes from the java.time packages.
LocalDateTime cannot represent a moment
You are using LocalDateTime to track a moment, which it cannot. By definition, that class does not represent a point on the time line. That class has a date and a time-of-day but intentionally lacks the context of a time zone or offset-from-UTC. Calling toLocalDateTime strips away vital information about zone/offset.
Tools lie
You are likely getting confused by the well-intentioned but unfortunate behavior of many tools to dynamically apply a time zone while generating text to represent the date-time value retrieved from the database. You can avoid this by using Java & JDBC to get the pure unadulterated value from the database.
TIMESTAMP WITH TIME ZONE
You failed to disclose the exact data type of your column in your database. If you are trying to track a moment, use a data type akin to the SQL-standard type TIMESTAMP WITH TIME ZONE. In MySQL 8 that would, apparently, be the TIMESTAMP type according to this doc. (I am a Postgres guy, not a MySQL user.)
In JDBC 4.2 and later, we can exchange java.time objects with the database. So no need to over touch java.sql.Timestamp again.
Unfortunately, the JDBC spec oddly chose to not require support for Instant (a moment in UTC) nor forZonedDateTime(a moment as seen in some particular time zone). The spec does require support for [OffsetDateTime`]2.
Tip: Learn to work in UTC for the most part. Adjust into a time zone only when required by business logic or for presentation to the user.
OffsetDateTime odt = OffsetDateTime.now( ZoneOffset.UTC ) ; // Capture current moment in UTC.
Write to the database via a prepared statement.
myPreparedStatement.setObject( … , odt ) ;
Retrieval.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ;
View that moment through the wall-clock time used by the people of some particular region (a time zone).
ZoneId z = ZoneId.of( "Asia/Kolkata" ) ;
ZonedDateTime zdt = odt.atZoneSameInstant( z ) ;
Related
I'm working on a java web project that uses:
Hibernate 5.2.2/JPA 2.0+ MySQL5InnoDBDialect
MySQL 5.6.15-innoDB (on EasyPHP/PHPMyAdmin) + JDBC connector 6.0.4
Joda time API 2.9.4 + Fasterxml jackson API 2.8.3
I'm facing a problem on inserting Time data on database. everytime i put a row, i get a +1H value on time column!
Attribute on Java:
#JsonFormat(shape=JsonFormat.Shape.STRING, pattern="HH:mm")
#Column(name = "RES_DUREE", nullable = false)
#Temporal(TemporalType.TIME) private Date resDuree;
Attribute on SQL:
RES_DUREE TIME NOT NULL;
EDIT (After Adrian Shum's Comment):
Connection line:
jdbc.url =
jdbc:mysql://localhost:3306/mydb?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
I do use UTC but it still 1H+ .
Any suggestion will help, thanks.
Normally it is caused by server and DB time zone mismatch.
In brief, java.util.Date does not contain Timezone information. Conceptually you can treat it as simply representing Date + Time (similar to what JODA/JSR310 LocalDateTime is doing). Therefore if your app server is UTC+10, and your DB is UTC+2, when you save a date of "2016-10-13 10:11:12", although your app server is treating it as "2016-10-13 10:11:12 +10:00", it is simply passing "2016-10-13 10:11:12" to DB. Given DB is UTC+2, it is thinking the time actually means "2016-10-13 10:11:12 +02:00". Situation become more messy if your JDBC connection is claimed to be "UTC+10", most DB is going to "smartly" translate "2016-10-13 10:11:12 +02:00" to "2016-10-13 18:11:12 +10:00" which caused weird time stored and retrieved.
You may diagnose by tracing the SQL (and the actual value used) for corresponding inserts and select. You should see discrepancies between the values, vs the value stored in table. Such tracing can be done by misc way, e.g.
Older version of Hibernate can show the parameter used in prepared statement by turning on proper logger
You may use tools like JdbcDsLog (Disclaimer: I am maintainer for a fork of JbdcDsLog at http://github.com/adrianshum/jdbcdslog)
There is probably tools in DBMS side to trace incoming queries.
Best way to solve is to make sure everything is in the same timezone, and the most rational choice for timezone is UTC.
I am using grails 2.4.4 for building a restful API. I am having a lot of trouble figuring out grails default time-zone settings.
The premise of the problem is I am using default grails setting without using and dateBinders in config.groovy and the timezone of my both mysql and application server is in NPT.
So I am posting a json something like
{"clusterStartTime":"2015-10-15T17:45:00Z"}
Now this is a ISO 8601 format. So how can this date format represent my timezone so grails can interpret it ??
In database it is being saved as 2015-10-15 17:45:00.
But when I get this resource I am getting :
{"clusterStartTime": "2015-10-15T12:00:00Z"}
So I am getting a time with offset of 5:45 which is NPT offset. So how is grails doing all of this ??
So in production I have a server in NPT timezone so how can I save all my dates in UTC timezone and spit out dates in UTC timezone using grails only without setting Java enviroment variables ??
In your bootstrap.groovy set the timezone like this:
TimeZone.setDefault(TimeZone.getTimeZone("Asia/Katmandu"))
NTP is not supported directly. To get a list of supported timezones see the output of TimeZone.getAvailableIDs().
In order to interpret correctly any ISO 8601 date you have to use a good parser. The one provided by javax.xml.bind.DatatypeConverter should be good enough. The method parseDateTime expects a lexicalXSDDateTime, which must be ISO 6801 compliant.
Therefore try this:
def cal = javax.xml.bind.DatatypeConverter.parseDateTime('2015-10-15T17:45:00Z')
def d = cal.getTime()
def t1 = new TestClass(myDate: d).save(failOnError:true)
println t1.myDate //Thu Oct 15 17:45:00 NPT 2015
The date gets saved on the database (e.g. in a DATETIME field) considering the time zone you defined, so it should be correct when you retrieve it.
I have a small Django project that imports data dumps from MongoDB into MySQL. Inside these Mongo dumps are dates stored in epoch time. I would expect epoch time to be the same regardless of time zone but what I am seeing is that the Django TIME_ZONE setting has an effect on the data created in MySQL.
I have been testing my database output with the MySQL UNIX_TIMESTAMP function. If I insert a date with the epoch of 1371131402880 (this includes milliseconds) I have my timezone set to 'America/New_York', UNIX_TIMESTAMP gives me 1371131402, which is the same epoch time excluding milliseconds. However if I set my timezone to 'America/Chicago' I get 1371127802.
This is my code to convert the epoch times into Python datetime objects,
from datetime import datetime
from django.utils.timezone import utc
secs = float(epochtime) / 1000.0
dt = datetime.fromtimestamp(secs)
I tried to fix the issue by putting an explict timezone on the datetime object,
# epoch time is in UTC by default
dt = dt.replace(tzinfo=utc)
PythonFiddle for the code
I've tested this Python code in isolation and it gives me the expected results. However it does not give the correct results after inserting these object into MySQL through a Django model DateTimeField field.
Here is my MySQL query,
SELECT id, `date`, UNIX_TIMESTAMP(`date`) FROM table
I test this by comparing the unix timestamp column in the result of this query against the MongoDB JSON dumps to see if the epoch matches.
What exactly is going on here? Why should timezone have any effect on epoch times?
Just for reference, I am using Django 1.5.1 and MySQL-python 1.2.4. I also have the Django USE_TZ flag set to true.
I am no python or Django guru, so perhaps someone can answer better than me. But I will take a guess at it anyway.
You said that you were storing it in a Django DateTimeField, which according to the documents you referenced, stores it as a Python datetime.
Looking at the docs for datetime, I think the key is understanding the difference between "naive" and "aware" values.
And then researching further, I came across this excellent reference. Be sure the read the second section, "Naive and aware datetime objects". That gives a bit of context to how much of this is being controlled by Django. Basically, by setting USE_TZ = true, you are asking Django to use aware datetimes instead of naive ones.
So then I looked back at you question. You said you were doing the following:
dt = datetime.fromtimestamp(secs)
dt = dt.replace(tzinfo=utc)
Looking at the fromtimestamp function documentation, I found this bit of text:
If optional argument tz is None or not specified, the timestamp is converted to the platform’s local date and time, and the returned datetime object is naive.
So I think you could do this:
dt = datetime.fromtimestamp(secs, tz=utc)
Then again, right below that function, the docs show utcfromtimestamp function, so maybe it should be:
dt = datetime.utcfromtimestamp(secs)
I don't know enough about python to know if these are equivalent or not, but you could try and see if either makes a difference.
Hopefully one of these will make a difference. If not, please let me know. I'm intimately familiar with date/time in JavaScript and in .Net, but I'm always interested in how these nuances play out differently in other platforms, such as Python.
Update
Regarding the MySQL portion of the question, take a look at this fiddle.
CREATE TABLE foo (`date` DATETIME);
INSERT INTO foo (`date`) VALUES (FROM_UNIXTIME(1371131402));
SET TIME_ZONE="+00:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
SET TIME_ZONE="+01:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
Results:
DATE UNIX_TIMESTAMP(`DATE`)
June, 13 2013 13:50:02+0000 1371131402
June, 13 2013 13:50:02+0000 1371127802
It would seem that the behavior of UNIX_TIMESTAMP function is indeed affected by the MySQL TIME_ZONE setting. That's not so surprising, since it's in the documentation. What's surprising is that the string output of the datetime has the same UTC value regardless of the setting.
Here's what I think is happening. In the docs for the UNIX_TIMESTAMP function, it says:
date may be a DATE string, a DATETIME string, a TIMESTAMP, or a number in the format YYMMDD or YYYYMMDD.
Note that it doesn't say that it can be a DATETIME - it says it can be a DATETIME string. So I think the actual value being implicitly converted to a string before being passed into the function.
So now look at this updated fiddle that converts explicitly.
SET TIME_ZONE="+00:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
SET TIME_ZONE="+01:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
Results:
DATE CONVERT(`DATE`, CHAR) UNIX_TIMESTAMP(CONVERT(`DATE`, CHAR))
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371131402
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371127802
You can see that when it converts to character data, it strips away the offset. So of course, it makes sense now that when UNIX_TIMESTAMP takes this value as input, it is assuming the local time zone setting and thus getting a different UTC timestamp.
Not sure if this will help you or not. You need to dig more into exactly how Django is calling MySQL for both the read and the write. Does it actually use the UNIX_TIMESTAMP function? Or was that just what you did in testing?
I'm currently working with a rails 3.0.x app that's recently been upgraded from rails 2.x. Recently, it has been discovered that our timestamps in MySQL have changed from being written in PST timezone offset to a UTC offset as a result of this migration.
After examining the problem it's been decided we want to keep all our timestamps in a UTC format going forward. Unfortunately, we need to go through all our old records that have timestamps in PST and convert them to UTC (a somewhat complicated process). I'd rather not re-invent the wheel if at all possible, which leads me to my question:
Has anyone written a utility to handle conversion of PST timestamps to UTC timestamps for a MySQL database?
This is why you always save times in UTC and render them in the user's local time if required. Sorry you had to find out the hard way!
What you could do is make a note of the id values where the transition occurred, and then adjust all timestamps prior to that interval with the CONVERT_TZ() method in MySQL.
That would look something like this for each table:
# List of maximum ID to adjust
max_id = {
'examples' => 100,
}
c = ActiveRecord::Base.connection
c.tables.each do |table|
# Set your specific time-zones as required, PST used as an example here.
timestamp_updates = c.columns(table).select do |col|
col.type == :datetime
end.collect(&:name).collect do |col|
"`#{col}`=CONVERT_TZ(`#{col}`, 'PST', 'UTC')"
end
next if (timestamp_updates.empty?)
c.execute("UPDATE `#{table}` SET #{timestamp_updates.join(',') WHERE id<=%d" % max_id[table])
end
I have a MySQL Server set to UTC (##global.time_zone = '+00:00') and a table with a DATETIME column in which I store dates in UTC. I'm having problems getting UTC dates to come through when I call a stored procedure via JDBC. Example:
java.util.Date now = new java.util.Date();
sproc = conn.prepareCall("{call TzTestInsert(?)}");
sproc.setTimestamp(1, new java.sql.Timestamp(now.getTime()), Calendar.getInstance(TimeZone.getTimeZone("GMT+00:00")));
sproc.execute();
The TzTestInsert sproc simply takes a DATETIME and inserts it into the table.
I'd expect the database to now hold my current time in UTC, but in fact it holds the current time for my timezone.
If I change the sproc to take a string it works...
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
...
sproc.setString(1, dateFormat.format(now));
but I'd rather use the correct type in the sproc.
Also works if I bypass the sproc, but again not my preferred solution...
String sql = "INSERT INTO TzTest VALUES('" + dateFormat.format(now) + "') ;
With the original sproc I have the same issue if I use a TIMESTAMP datatype in the sproc and table, which isn't surprising with the server in UTC since any timezone conversions specific to MySQL TIMESTAMP should be noops.
Calling the sproc from a MySQL Workbench connection works fine, e.g.
CALL TzTestInsert(UTC_TIMESTAMP());
Seems like the problem is in JDBC. I've looked at the various timezone connection parameters and haven't found any that make a difference.
I must be missing something basic - lots of people do this, right?
Solution was to pass the JDBC driver "useLegacyDatetimeCode=false". See mysql bug http://bugs.mysql.com/bug.php?id=15604
Looks like they left the old code in the driver for backwards compatibility.