How to save and retain Date milliseconds (MySQL) - mysql

I need to generate the current Date in vb.net that's have also the millisecond. Actually I use:
Date.Now
But this returns a simple date as 28/12/2015 16:53, I want also the millisecond something like:
28/12/2015 16:53:48640864
I tried with:
Public Shared Function MillisecondsDate()
Return Date.Now.ToString("HH:mm:ss.fffffff")
End Function
but this returns a bad result:
16:53:56.9884043
I need a format compatible with MySql for an accurate lastUpdated field. Any ideas?

From comments: my db field is a timestamp and I fix temp using a varchar instead a datetime or timestamp
A TimeStamp column isnt quite the same as a DateTime column. According to the Documentation:
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.)
It is something you may never see, even MySQL WorkBench UI converts it back.
Fractional Seconds
The issue of Fractional Seconds is something else. When defining a column as DateTime or TimeStamp, fractional seconds are dropped by default, for compatibility with older versions.
Rather than defining any sort of Date column as string/text/varchar - and of course having to parse it back - you can define either type to retain fractional seconds. In MySQL 5.6.4+, you can use: DATETIME(n) or TIMESTAMP(n) (yes, thats why the parens show in the UI dropdown list). Where n is the number of fractional digits to store (0-6). For .NET, 3 would equate to milliseconds. After that, the WorkBench UI shows the milliseconds as well:
StartDate is DateTime(0) (no fractionals)
LastUpdated is TimeStamp(3)
Foo is TimeStamp(6)
Trigger
A LastUpdated column would seem best managed by the database so you don't have to do so via code -- or forget to do so!. This is pretty simple using a default value and a Before_Update trigger:
Define the column as TIMESTAMP(3)
Specify CURRENT_TIMESTAMP(3) as the default value
ADD COLUMN LastUpdated TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) COMMENT '';
This would automatically set the LastUpdated value on new items added to the current time (UTC etc) with milliseconds (TIMESTAMP(6) might be better, but the question only frets about ms). Be sure to specify the same number of digits in the default.
Then, rather than passing DateTime.Now manually (or converting to string) when updating each record, add a trigger to update the column for you. In MySql Workbench:
CREATE DEFINER=`root`#`localhost` TRIGGER `test`.`demo_BEFORE_INSERT` BEFORE INSERT ON `demo` FOR EACH ROW
BEGIN
SET new.LastUpdated := now(3);
END
Most of that is boilerplate that the UI Tool creates. You just need to type:
SET new.LastUpdated := now(3);
For this type of column, you'd can add the same thing on the BEFORE_INSERT trigger in place of a default value. In MySQL WorkBench, click the "Triggers" tab when viewing the table definitions - just add the SET statement.
Test Code
This shows the fractional milliseconds making the round trip from code to db and back, and shows that the trigger works (uses the table shown above):
Dim Usql = "UPDATE Demo SET Foo = #p1 WHERE Id=5"
Dim Ssql = "SELECT Name, StartDate, Foo, LastUpdated FROM Demo WHERE Id=5"
Dim dtVar As DateTime = DateTime.Now
Console.WriteLine("DT ms in code var: {0}", dtVar.Millisecond)
Using dbcon = GetMySQLConnection()
dbcon.Open()
Using cmd As New MySqlCommand(Usql, dbcon)
cmd.Parameters.AddWithValue("#p1", dtVar)
cmd.ExecuteNonQuery()
End Using
Using cmd As New MySqlCommand(Ssql, dbcon)
Using rdr As MySqlDataReader = cmd.ExecuteReader
If rdr.HasRows Then
rdr.Read()
Dim tempDT = rdr.GetMySqlDateTime(1) ' DateTime(0)
Console.WriteLine("DT.MS from DB {0}", tempDT.Millisecond)
Dim mydt = rdr.GetMySqlDateTime(2) ' TimeStamp(6)
Console.WriteLine("Micro from DB {0} ", mydt.Microsecond)
' either get method retains the ms TimeStamp(3)
Dim lstupD = Convert.ToDateTime(rdr("lastUpdated"))
Console.WriteLine("MS from trigger {0}", lstupD.Millisecond)
End If
End Using
End Using
End Using
Result:
DT ms in code var: 615
DT.MS from DB 0
Micro from DB 615413
MS from trigger 615
The columns defined to store fractional seconds do so down to microseconds (ticks). Note that the Update SQL doesn't reference the LastUpdated column, yet it is updated courtesy of the trigger. If you are step-debugging and using a variable as above the trigger time can vary from the others because time is passing as you step through.

Try this
Public Shared Function MillisecondsDate()
Return Date.Now.ToString("dd/MM/yyyy HH:mm:ss.fff")
End Function

Try this function:
Public Shared Function MillisecondsDate()
Return DateTime.UtcNow.ToString("dd/MM/yyyy HH:mm:ss.fff")
End Function

Related

Google Apps Script - MySQL data import using JDCB does not work with Date 0000-00-00 [duplicate]

I have a database table containing dates
(`date` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00').
I'm using MySQL. From the program sometimes data is passed without the date to the database. So, the date value is auto assigned to 0000-00-00 00:00:00
when the table data is called with the date column it gives error
...'0000-00-00 00:00:00' can not be represented as java.sql.Timestamp.......
I tried to pass null value to the date when inserting data, but it gets assign to the current time.
Is there any way I can get the ResultSet without changing the table structure?
You can use this JDBC URL directly in your data source configuration:
jdbc:mysql://yourserver:3306/yourdatabase?zeroDateTimeBehavior=convertToNull
Whether or not the "date" '0000-00-00" is a valid "date" is irrelevant to the question.
"Just change the database" is seldom a viable solution.
Facts:
MySQL allows a date with the value of zeros.
This "feature" enjoys widespread use with other languages.
So, if I "just change the database", thousands of lines of PHP code will break.
Java programmers need to accept the MySQL zero-date and they need to put a zero date back into the database, when other languages rely on this "feature".
A programmer connecting to MySQL needs to handle null and 0000-00-00 as well as valid dates. Changing 0000-00-00 to null is not a viable option, because then you can no longer determine if the date was expected to be 0000-00-00 for writing back to the database.
For 0000-00-00, I suggest checking the date value as a string, then changing it to ("y",1), or ("yyyy-MM-dd",0001-01-01), or into any invalid MySQL date (less than year 1000, iirc). MySQL has another "feature": low dates are automatically converted to 0000-00-00.
I realize my suggestion is a kludge. But so is MySQL's date handling.
And two kludges don't make it right. The fact of the matter is, many programmers will have to handle MySQL zero-dates forever.
Append the following statement to the JDBC-mysql protocol:
?zeroDateTimeBehavior=convertToNull&autoReconnect=true&characterEncoding=UTF-8&characterSetResults=UTF-8
for example:
jdbc:mysql://localhost/infra?zeroDateTimeBehavior=convertToNull&autoReconnect=true&characterEncoding=UTF-8&characterSetResults=UTF-8
Instead of using fake dates like 0000-00-00 00:00:00 or 0001-01-01 00:00:00 (the latter should be accepted as it is a valid date), change your database schema, to allow NULL values.
ALTER TABLE table_name MODIFY COLUMN date TIMESTAMP NULL
As an exteme turnaround, when you cannot do an alter to your date column or to update the values, or while these modifications take place, you can do a select using a case/when.
SELECT CASE ModificationDate WHEN '0000-00-00 00:00:00' THEN '1970-01-01 01:00:00' ELSE ModificationDate END AS ModificationDate FROM Project WHERE projectId=1;
you can try like This
ArrayList<String> dtlst = new ArrayList<String>();
String qry1 = "select dt_tracker from gs";
Statement prepst = conn.createStatement();
ResultSet rst = prepst.executeQuery(qry1);
while(rst.next())
{
String dt = "";
try
{
dt = rst.getDate("dt_tracker")+" "+rst.getTime("dt_tracker");
}
catch(Exception e)
{
dt = "0000-00-00 00:00:00";
}
dtlst.add(dt);
}
I wrestled with this problem and implemented the URL concatenation solution contributed by #Kushan in the accepted answer above. It worked in my local MySql instance. But when I deployed my Play/Scala app to Heroku it no longer would work. Heroku also concatenates several args to the DB URL that they provide users, and this solution, because of Heroku's use concatenation of "?" before their own set of args, will not work. However I found a different solution which seems to work equally well.
SET sql_mode = 'NO_ZERO_DATE';
I put this in my table descriptions and it solved the problem of
'0000-00-00 00:00:00' can not be represented as java.sql.Timestamp
There was no year 0000 and there is no month 00 or day 00. I suggest you try
0001-01-01 00:00:00
While a year 0 has been defined in some standards, it is more likely to be confusing than useful IMHO.
just cast the field as char
Eg: cast(updatedate) as char as updatedate
I know this is going to be a late answer, however here is the most correct answer.
In MySQL database, change your timestamp default value into CURRENT_TIMESTAMP. If you have old records with the fake value, you will have to manually fix them.
You can remove the "not null" property from your column in mysql table if not necessary. when you remove "not null" property no need for "0000-00-00 00:00:00" conversion and problem is gone.
At least worked for me.
I believe this is help full for who are getting this below Exception on to pumping data through logstash
Error: logstash.inputs.jdbc - Exception when executing JDBC query {:exception=>#}
Answer:jdbc:mysql://localhost:3306/database_name?zeroDateTimeBehavior=convertToNull"
or if you are working with mysql

Node.js Updating MySql timestamp

So i do a simple query like so:
connection.query("UPDATE workers SET timestamp='"+thedate+"' WHERE id = " + id,function(err,upres){
connection.release();
if(!err) {
console.log('updated record')
console.log(upres);
}
})
console.log reveals the data format as: 2015-04-02 19:29:14
And if i debug the SQL statement, that turns out to be:
UPDATE workers SET timestamp='2015-04-02 21:31:16' WHERE id = 3;
However, when i list the data, the output is:
[{"id":3,"worker":"John Doe","timestamp":"2015-04-01T22:00:00.000Z","duration":30}]
This is way off compared to the time that is being reported?
What is causing this?
You do not know how MySQL is turning your VARCHAR into a date. There are a lot of configuration options. It would be better to use the STR_TO_DATE function to circumvent all of the assumptions. Here is a link to the docs for STR_TO_DATE().
As a side note, I would strongly recommend using prepared statements as a way to safeguard your application against errors and sql injection.
EDITS:
In regards to your questions, the column could be DATETIME, but your value you are assigning is a VARCHAR
'UPDATE workers SET timestamp = ? WHERE id = ?', ['4/2/2015 3:00:00 PM', 3'], [callBackFunction]
Based on what you said about the conversion not working, I am suspicious about the data type for the timestamp column.
SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE NAME = 'workers'
A statement like that should give you all of the information about that column. You could also find this in a GUI, if you have access. There are three different date types in MySQL date, datetime, or timestamp. This is most likely a DATE column, that will not be able to hold the time.

MySqlConversionException issue

i want to insert date now into mysql database
i need to convert the format but it didn't worked
this is my code
Dim now As Datetime = Datetime.Now.ToString("yyyy-MM-dd")
but it didn't worked either
as i google this i found out that the problem is because the parameter in my connection string AllowDateTimeZero=True
but when i tried to add it in my conn string, it stil didn't worked. could someone help me about this
i've spent 2 hours trying to fix this, but it's not worked..
thanks before
In VB, a DateTime data type holds the number if TICKs that have occured since 1/1/0001 12:00:00 AM. When you convert the DateTime.Now into a string, then put it into another DateTime, VB must convert the string "1985-12-31" into the number of ticks.
The code you posted...
Dim now As Datetime = Datetime.Now.ToString("yyyy-MM-dd")
...does nothing except introduce the possibility of issues due to culture difference in DateTime.
What you should have done is posted the code you use to insert the data into MySQL. If you are using a command and a non-parameterized INSERT:
INSERT INTO MyTable (MyDate) VALUES (" & Datetime.Now.ToString("yyyy-MM-dd") & ")
If it is parameterized, as it should be:
cmd.AddParameterWithValue("p1",Datetime.Now)
If you are using the EF or some type of DBML:
MyTable.MyDate = Datetime.Now

Why does the Django time zone setting effect epoch time?

I have a small Django project that imports data dumps from MongoDB into MySQL. Inside these Mongo dumps are dates stored in epoch time. I would expect epoch time to be the same regardless of time zone but what I am seeing is that the Django TIME_ZONE setting has an effect on the data created in MySQL.
I have been testing my database output with the MySQL UNIX_TIMESTAMP function. If I insert a date with the epoch of 1371131402880 (this includes milliseconds) I have my timezone set to 'America/New_York', UNIX_TIMESTAMP gives me 1371131402, which is the same epoch time excluding milliseconds. However if I set my timezone to 'America/Chicago' I get 1371127802.
This is my code to convert the epoch times into Python datetime objects,
from datetime import datetime
from django.utils.timezone import utc
secs = float(epochtime) / 1000.0
dt = datetime.fromtimestamp(secs)
I tried to fix the issue by putting an explict timezone on the datetime object,
# epoch time is in UTC by default
dt = dt.replace(tzinfo=utc)
PythonFiddle for the code
I've tested this Python code in isolation and it gives me the expected results. However it does not give the correct results after inserting these object into MySQL through a Django model DateTimeField field.
Here is my MySQL query,
SELECT id, `date`, UNIX_TIMESTAMP(`date`) FROM table
I test this by comparing the unix timestamp column in the result of this query against the MongoDB JSON dumps to see if the epoch matches.
What exactly is going on here? Why should timezone have any effect on epoch times?
Just for reference, I am using Django 1.5.1 and MySQL-python 1.2.4. I also have the Django USE_TZ flag set to true.
I am no python or Django guru, so perhaps someone can answer better than me. But I will take a guess at it anyway.
You said that you were storing it in a Django DateTimeField, which according to the documents you referenced, stores it as a Python datetime.
Looking at the docs for datetime, I think the key is understanding the difference between "naive" and "aware" values.
And then researching further, I came across this excellent reference. Be sure the read the second section, "Naive and aware datetime objects". That gives a bit of context to how much of this is being controlled by Django. Basically, by setting USE_TZ = true, you are asking Django to use aware datetimes instead of naive ones.
So then I looked back at you question. You said you were doing the following:
dt = datetime.fromtimestamp(secs)
dt = dt.replace(tzinfo=utc)
Looking at the fromtimestamp function documentation, I found this bit of text:
If optional argument tz is None or not specified, the timestamp is converted to the platform’s local date and time, and the returned datetime object is naive.
So I think you could do this:
dt = datetime.fromtimestamp(secs, tz=utc)
Then again, right below that function, the docs show utcfromtimestamp function, so maybe it should be:
dt = datetime.utcfromtimestamp(secs)
I don't know enough about python to know if these are equivalent or not, but you could try and see if either makes a difference.
Hopefully one of these will make a difference. If not, please let me know. I'm intimately familiar with date/time in JavaScript and in .Net, but I'm always interested in how these nuances play out differently in other platforms, such as Python.
Update
Regarding the MySQL portion of the question, take a look at this fiddle.
CREATE TABLE foo (`date` DATETIME);
INSERT INTO foo (`date`) VALUES (FROM_UNIXTIME(1371131402));
SET TIME_ZONE="+00:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
SET TIME_ZONE="+01:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
Results:
DATE UNIX_TIMESTAMP(`DATE`)
June, 13 2013 13:50:02+0000 1371131402
June, 13 2013 13:50:02+0000 1371127802
It would seem that the behavior of UNIX_TIMESTAMP function is indeed affected by the MySQL TIME_ZONE setting. That's not so surprising, since it's in the documentation. What's surprising is that the string output of the datetime has the same UTC value regardless of the setting.
Here's what I think is happening. In the docs for the UNIX_TIMESTAMP function, it says:
date may be a DATE string, a DATETIME string, a TIMESTAMP, or a number in the format YYMMDD or YYYYMMDD.
Note that it doesn't say that it can be a DATETIME - it says it can be a DATETIME string. So I think the actual value being implicitly converted to a string before being passed into the function.
So now look at this updated fiddle that converts explicitly.
SET TIME_ZONE="+00:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
SET TIME_ZONE="+01:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
Results:
DATE CONVERT(`DATE`, CHAR) UNIX_TIMESTAMP(CONVERT(`DATE`, CHAR))
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371131402
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371127802
You can see that when it converts to character data, it strips away the offset. So of course, it makes sense now that when UNIX_TIMESTAMP takes this value as input, it is assuming the local time zone setting and thus getting a different UTC timestamp.
Not sure if this will help you or not. You need to dig more into exactly how Django is calling MySQL for both the read and the write. Does it actually use the UNIX_TIMESTAMP function? Or was that just what you did in testing?

MySQL - Passing UTC timestamps to sprocs via JDBC

I have a MySQL Server set to UTC (##global.time_zone = '+00:00') and a table with a DATETIME column in which I store dates in UTC. I'm having problems getting UTC dates to come through when I call a stored procedure via JDBC. Example:
java.util.Date now = new java.util.Date();
sproc = conn.prepareCall("{call TzTestInsert(?)}");
sproc.setTimestamp(1, new java.sql.Timestamp(now.getTime()), Calendar.getInstance(TimeZone.getTimeZone("GMT+00:00")));
sproc.execute();
The TzTestInsert sproc simply takes a DATETIME and inserts it into the table.
I'd expect the database to now hold my current time in UTC, but in fact it holds the current time for my timezone.
If I change the sproc to take a string it works...
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
...
sproc.setString(1, dateFormat.format(now));
but I'd rather use the correct type in the sproc.
Also works if I bypass the sproc, but again not my preferred solution...
String sql = "INSERT INTO TzTest VALUES('" + dateFormat.format(now) + "') ;
With the original sproc I have the same issue if I use a TIMESTAMP datatype in the sproc and table, which isn't surprising with the server in UTC since any timezone conversions specific to MySQL TIMESTAMP should be noops.
Calling the sproc from a MySQL Workbench connection works fine, e.g.
CALL TzTestInsert(UTC_TIMESTAMP());
Seems like the problem is in JDBC. I've looked at the various timezone connection parameters and haven't found any that make a difference.
I must be missing something basic - lots of people do this, right?
Solution was to pass the JDBC driver "useLegacyDatetimeCode=false". See mysql bug http://bugs.mysql.com/bug.php?id=15604
Looks like they left the old code in the driver for backwards compatibility.