Timestamp with a millisecond precision: How to save them in MySQL - mysql

I have to develop a application using MySQL and I have to save values like "1412792828893" which represent a timestamp but with a precision of a millisecond. That is, the amount of milliseconds since 1.1.1970. I declare the row as timestamp but unfortunately this didn't work. All values are set to 0000-00-00 00:00:00
CREATE TABLE IF NOT EXISTS `probability` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`segment_id` int(11) NOT NULL,
`probability` float NOT NULL,
`measured_at` timestamp NOT NULL,
`provider_id` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ;
How should be the declaration in order to be able to save timestamp values with this precision?

You need to be at MySQL version 5.6.4 or later to declare columns with fractional-second time datatypes. Not sure you have the right version? Try SELECT NOW(3). If you get an error, you don't have the right version.
For example, DATETIME(3) will give you millisecond resolution in your timestamps, and TIMESTAMP(6) will give you microsecond resolution on a *nix-style timestamp.
Read this: https://dev.mysql.com/doc/refman/8.0/en/fractional-seconds.html
NOW(3) will give you the present time from your MySQL server's operating system with millisecond precision.
If you have a number of milliseconds since the Unix epoch, try this to get a DATETIME(3) value
FROM_UNIXTIME(ms * 0.001)
Javascript timestamps, for example, are represented in milliseconds since the Unix epoch.
(Notice that MySQL internal fractional arithmetic, like * 0.001, is always handled as IEEE754 double precision floating point, so it's unlikely you'll lose precision before the Sun becomes a white dwarf star.)
If you're using an older version of MySQL and you need subsecond time precision, your best path is to upgrade. Anything else will force you into doing messy workarounds.
If, for some reason you can't upgrade, you could consider using BIGINT or DOUBLE columns to store Javascript timestamps as if they were numbers. FROM_UNIXTIME(col * 0.001) will still work OK. If you need the current time to store in such a column, you could use UNIX_TIMESTAMP() * 1000

CREATE TABLE fractest( c1 TIME(3), c2 DATETIME(3), c3 TIMESTAMP(3) );
INSERT INTO fractest VALUES
('17:51:04.777', '2018-09-08 17:51:04.777', '2018-09-08 17:51:04.777');

please create the table like this by mentioning the length (length can be whatever digit count that you want by milliseconds) of timestamp timestamp(2) or timestamp(3) or timestamp(5) likewise. the Mysql version should be 5.6 or above.
https://dev.mysql.com/doc/refman/5.6/en/fractional-seconds.html
CREATE TABLE IF NOT EXISTS `probability` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`measured_at` timestamp(2) NOT NULL,
PRIMARY KEY (`id`)
) ;
and then pass the timestamp through Java.
statement.setTimestamp(2, new Timestamp(new Date().getTime()))

You can use BIGINT as follows:
CREATE TABLE user_reg (
user_id INT NOT NULL AUTO_INCREMENT,
identifier INT,
phone_number CHAR(11) NOT NULL,
verified TINYINT UNSIGNED NOT NULL,
reg_time BIGINT,
last_active_time BIGINT,
PRIMARY KEY (user_id),
INDEX (phone_number, user_id, identifier)
);

Related

How to update timestamp column in MySQL with incorrect timestamp?

I have a database (MySQL 8.0) with four timestamp columns, using the default timestamp settings, i.e. no fraction of seconds. I would like to start using fractions of a second, probably two decimals, so TIMESTAMP(2). The process that generates the data does not always provide a timestamp to column timestamp_column_3 (just renamed the columns as timestamp_column_1 to timestamp_column_4 here) and thus there are many '0000-00-00 00:00:00' in timestamp_column_3. When I tried converting the
timestamp_column_1 by running the following query:
ALTER TABLE table_name MODIFY COLUMN timestamp_column TIMESTAMP(2);
I get the following response:
Error Code: 1292. Incorrect datetime value: '0000-00-00 00:00:00' for column 'timestamp_column_3' at row 74608.
So, two questions:
Why does timestamp_column_3 interfere with my altering of column timestamp_column_1?
How do I proceed to convert all four columns to datatype TIMESTAMP(2)?
I looked around and found this answer to a similar question. But I'd rather not modify the mode of the database as I'm not very inexperienced and this is a production database. Is there a way to adjust the timestamps in the column to the minimum allowed (I assume this is 1970-01-01 00:00:00). I don't really care about the value. Also, I don't understand why the insertion process is allowed to insert these incorrect values as the mode is the following:
STRICT_TRANS_TABLES,
NO_ZERO_IN_DATE,
NO_ZERO_DATE,
ERROR_FOR_DIVISION_BY_ZERO,
NO_ENGINE_SUBSTITUTION
The incorrect values doesn't really matter as we know they don't "exist" but of course it would be nice to have everything correct. What would then be the "correct" value instead of a ZERO_DATE?
-- EDIT -- Add some information
Version: 8.0.18-google
Table:
CREATE TABLE `event` (
`c1` int(11) DEFAULT NULL,
`c2` varchar(32) NOT NULL,
`c3` varchar(32) NOT NULL,
`c4` varchar(64) NOT NULL,
`c5` varchar(64) NOT NULL,
`c6` varchar(64) NOT NULL,
`c7` varchar(64) NOT NULL,
`c8` varchar(64) NOT NULL,
`c9` varchar(128) DEFAULT NULL,
`timestamp_column_1` timestamp NOT NULL,
`timestamp_column_2` timestamp NOT NULL,
`timestamp_column_3` timestamp NULL DEFAULT NULL,
`timestamp_column_4` timestamp NOT NULL,
PRIMARY KEY (`c4`),
KEY `event-timestamp_column_1` (`timestamp_column_1`),
KEY `event-timestamp_column_3` (`timestamp_column_3`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
0, a, b, c, d, e, f, g, , 2021-02-19 07:45:30, 2021-02-19 07:45:29, 0000-00-00 00:00:00, 2021-06-03 20:11:45
The data is added through Google Storage CSV import function and timestamp_column_3 sometimes has no data in the CSV file, i.e. the column is just represented as ,, in the CSV.

Having a "Creation Date" column with default value at table creation

I want to have a column that will store the creation date of a row.
I'm using php and mysql but I don't think that matters.
I've looked for a series of answers about that but all of them seem to be for updating an existing table. Well surely there's one for what I'm looking for since it's a pretty basic question but I've yet to find it.
I've tried things with DEFAULT, CONSTRAINT but none of them allow me to create the table once added to my code. You could have the feeling that I'm not well versed in sql and you would not be wrong.
This creates the table, could you tell me what to add ?
CREATE TABLE IF NOT EXISTS artwork (
id_artwork int(4) NOT NULL AUTO_INCREMENT,
title varchar(50) NOT NULL,
creationDate DateTime(3),
CONSTRAINT PK_artwork PRIMARY KEY (id_artwork)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
I've tried the following with no success:
creationDate DateTime(3) DEFAULT GETDATE()
creationDate DateTime(3) DEFAULT (GETDATE())
creationDate DATETIME(3) DEFAULT (CURRENT_TIMESTAMP)
MySQL Version: 5.7.23 - MySQL Community Server (GPL)
From the MySQL documentation on initialization using DATETIME:
If a TIMESTAMP or DATETIME column definition includes an explicit fractional seconds precision value anywhere, the same value must be used throughout the column definition.
This means we'll have to carry forward your precision. I was able to get it to work on SQL Fiddle:
CREATE TABLE IF NOT EXISTS artwork (
id_artwork int(4) NOT NULL AUTO_INCREMENT,
title varchar(50) NOT NULL,
creationDate DateTime(3) DEFAULT CURRENT_TIMESTAMP(3),
CONSTRAINT PK_artwork PRIMARY KEY (id_artwork)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

how can i insert current timestamp in mysql with millisecond accuracy [duplicate]

I have to develop a application using MySQL and I have to save values like "1412792828893" which represent a timestamp but with a precision of a millisecond. That is, the amount of milliseconds since 1.1.1970. I declare the row as timestamp but unfortunately this didn't work. All values are set to 0000-00-00 00:00:00
CREATE TABLE IF NOT EXISTS `probability` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`segment_id` int(11) NOT NULL,
`probability` float NOT NULL,
`measured_at` timestamp NOT NULL,
`provider_id` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ;
How should be the declaration in order to be able to save timestamp values with this precision?
You need to be at MySQL version 5.6.4 or later to declare columns with fractional-second time datatypes. Not sure you have the right version? Try SELECT NOW(3). If you get an error, you don't have the right version.
For example, DATETIME(3) will give you millisecond resolution in your timestamps, and TIMESTAMP(6) will give you microsecond resolution on a *nix-style timestamp.
Read this: https://dev.mysql.com/doc/refman/8.0/en/fractional-seconds.html
NOW(3) will give you the present time from your MySQL server's operating system with millisecond precision.
If you have a number of milliseconds since the Unix epoch, try this to get a DATETIME(3) value
FROM_UNIXTIME(ms * 0.001)
Javascript timestamps, for example, are represented in milliseconds since the Unix epoch.
(Notice that MySQL internal fractional arithmetic, like * 0.001, is always handled as IEEE754 double precision floating point, so it's unlikely you'll lose precision before the Sun becomes a white dwarf star.)
If you're using an older version of MySQL and you need subsecond time precision, your best path is to upgrade. Anything else will force you into doing messy workarounds.
If, for some reason you can't upgrade, you could consider using BIGINT or DOUBLE columns to store Javascript timestamps as if they were numbers. FROM_UNIXTIME(col * 0.001) will still work OK. If you need the current time to store in such a column, you could use UNIX_TIMESTAMP() * 1000
CREATE TABLE fractest( c1 TIME(3), c2 DATETIME(3), c3 TIMESTAMP(3) );
INSERT INTO fractest VALUES
('17:51:04.777', '2018-09-08 17:51:04.777', '2018-09-08 17:51:04.777');
please create the table like this by mentioning the length (length can be whatever digit count that you want by milliseconds) of timestamp timestamp(2) or timestamp(3) or timestamp(5) likewise. the Mysql version should be 5.6 or above.
https://dev.mysql.com/doc/refman/5.6/en/fractional-seconds.html
CREATE TABLE IF NOT EXISTS `probability` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`measured_at` timestamp(2) NOT NULL,
PRIMARY KEY (`id`)
) ;
and then pass the timestamp through Java.
statement.setTimestamp(2, new Timestamp(new Date().getTime()))
You can use BIGINT as follows:
CREATE TABLE user_reg (
user_id INT NOT NULL AUTO_INCREMENT,
identifier INT,
phone_number CHAR(11) NOT NULL,
verified TINYINT UNSIGNED NOT NULL,
reg_time BIGINT,
last_active_time BIGINT,
PRIMARY KEY (user_id),
INDEX (phone_number, user_id, identifier)
);

mysql select distinct date takes FOREVER on database w/ 374 million rows

I have a MYSQL DB with table definition like this:
CREATE TABLE `minute_data` (
`date` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`open` decimal(10,2) DEFAULT NULL,
`high` decimal(10,2) DEFAULT NULL,
`low` decimal(10,2) DEFAULT NULL,
`close` decimal(10,2) DEFAULT NULL,
`volume` decimal(10,2) DEFAULT NULL,
`adj_close` varchar(45) DEFAULT NULL,
`symbol` varchar(10) NOT NULL DEFAULT '',
PRIMARY KEY (`symbol`,`date`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
It stores 1 minute data points from the stock market. The primary key is a combination of the symbol and date columns. This way I always have only 1 data point for each symbol at any time.
I am wondering why the following query takes so long that I can't even wait for it to finish:
select distinct date from test.minute_data where date >= "2013-01-01"
order by date asc limit 100;
However I can select count(*) from minute_data; and that finishes very quickly.
I know that it must have something to do with the fact that there are over 374 million rows of data in the table, and my desktop computer is pretty far from a super computer.
Does anyone know something I can try to speed up with query? Do I need to abandon all hope of using a MySQL table this big??
Thanks a lot!
When you have a composite index on 2 columns, like your (symbol, date) primary key, searching and grouping by a prefix of they key will be fast. But searching for something that doesn't include the first column in the index requires scanning all rows or using some other index.
You can either change your primary key to (date, symbol) if you don't usually need to search for symbol without date. Or you can add an additional index on date:
alter table minute_data add index (date)

Storing time in MySQL

I have data in the format of both "2013-01-17 18:46:47 -0800" and "1358477089" ...I'm wondering what is the best way to store this in a mysql db, that allows me to select results within a certain month, week, day etc.. using mysql's own functions.
Currently my create table code is like this.. the "timestamp" needs changing.
visible
CREATE TABLE IF NOT EXISTS `votes` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`from` varchar(32) NOT NULL,
`username` varchar(32) NOT NULL,
`address` varchar(16) NOT NULL,
`timestamp` varchar(32) NOT NULL,
PRIMARY KEY (`id`)) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Best way is to use MySQL built-in DATETIME type.
MySQL offers lots of function which will allow you to select results within a certain month, week, day, whatever you need.
See great list of functions here:
http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html
As hek2mgl and other guys mentioned, there is also TIMESTAMP.
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.)
If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored.
I preffer and advice you to use DATETIME.
If you use a timestamp your field should be an "integer" not a varchar. This provides better perfomance (for example if you use an index for this column).
If you do not need to have dates before 1970 I would suggest to use a timestamp, not a datetime. It is easier to use.
PHP
$timestamp = date('U');
MySQL
INSERT INTO table SET timestamp = UNIX_TIMESTAMP()