I've made a test. Here is a innodb table:
CREATE TABLE `test_UNIX_TIMESTAMP` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`datefrom` INT(11) NULL DEFAULT NULL,
PRIMARY KEY (`id`)
)
Here is an insert:
insert into test_UNIX_TIMESTAMP (datefrom) values
(UNIX_TIMESTAMP())
,(UNIX_TIMESTAMP())
,(UNIX_TIMESTAMP())
....many many
,(UNIX_TIMESTAMP());
Here is a query to check is it unique timestamps or not:
select distinct(datefrom) from test_UNIX_TIMESTAMP;
The results are:
After Affected rows: 106 400, we have 1 unique timestamp value in the table.
After Affected rows: 1 170 400 , we still have 1 unique timestamp value in the table.
MySQL server - it's remote SQL server (xeon 1270v3 with 32 ram, mysql have 4Gb cache).
Is "UNIX_TIMESTAMP()" value fixed for large insert queries? Or it's fixed for session? Or i'm just lucky?
The MySQL documentation isn't clear on this. However, it does draw a distinction between CURRENT_TIMESTAMP() (or NOW()) and SYSDATE() — the former is evaluated once per query, while the latter is evaluated at every invocation (and is presumably a bit slower).
You can demonstrate this for yourself by calling both either side of a SLEEP instruction:
mysql> select NOW() "a", SYSDATE() "b", SLEEP(2) "_", NOW() "c", SYSDATE() "d";
+---------------------+---------------------+---+---------------------+---------------------+
| a | b | _ | c | d |
+---------------------+---------------------+---+---------------------+---------------------+
| 2015-11-04 11:18:55 | 2015-11-04 11:18:55 | 0 | 2015-11-04 11:18:55 | 2015-11-04 11:18:57 |
+---------------------+---------------------+---+---------------------+---------------------+
1 row in set (2.01 sec)
As you can see, SYSDATE() increases during the query, while NOW() remains constant. It turns out that UNIX_TIMESTAMP() is also evaluated once per query:
mysql> select UNIX_TIMESTAMP(), SLEEP(2), UNIX_TIMESTAMP();
+------------------+----------+------------------+
| UNIX_TIMESTAMP() | SLEEP(2) | UNIX_TIMESTAMP() |
+------------------+----------+------------------+
| 1446635945 | 0 | 1446635945 |
+------------------+----------+------------------+
1 row in set (2.00 sec)
I guess it's the same as for the NOW() function:
NOW() returns a constant time that indicates the time at which the statement began to execute. (Within a stored function or trigger, NOW() returns the time at which the function or triggering statement began to execute.) This differs from the behavior for SYSDATE(), which returns the exact time at which it executes.
Unfortunately there's no specific documentation for other functions regarding this case. If you want to make sure, to always have the same value, just NOW() with UNIX_TIMESTAMP(), like
SELECT UNIX_TIMESTAMP(NOW());
Related
I want to show rows that have updated_at more than 3 hours ago. MySQL seems to be completely ignoring the ORDER BY clause. Any idea why?
Edit: as pointed out by Sebastian, this only occurs in certain timezones, like GMT+5 or GMT+8.
mysql> SET time_zone='+08:00';
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE DATABASE test1; USE test1;
Query OK, 1 row affected (0.01 sec)
Database changed
mysql> CREATE TABLE `boxes` (
-> `box_id` int unsigned NOT NULL AUTO_INCREMENT,
-> `updated_at` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
-> PRIMARY KEY (`box_id`)
-> ) ENGINE=InnoDB;
Query OK, 0 rows affected (0.01 sec)
mysql> INSERT INTO `boxes` (`box_id`, `updated_at`) VALUES
-> (1, '2020-08-22 05:25:35'),
-> (2, '2020-08-26 18:49:05'),
-> (3, '2020-08-23 03:28:30'),
-> (4, '2020-08-23 03:32:55');
Query OK, 4 rows affected (0.00 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> SELECT NOW();
+---------------------+
| NOW() |
+---------------------+
| 2020-08-26 20:49:59 |
+---------------------+
1 row in set (0.00 sec)
mysql> SELECT b.box_id, updated_at, (b.updated_at < NOW() - INTERVAL 3 HOUR) AS more_than_3hr
-> FROM boxes b
-> ORDER BY more_than_3hr DESC;
+--------+---------------------+---------------+
| box_id | updated_at | more_than_3hr |
+--------+---------------------+---------------+
| 1 | 2020-08-22 05:25:35 | 1 |
| 2 | 2020-08-26 18:49:05 | 0 | <--- WHY IS THIS HERE???
| 3 | 2020-08-23 03:28:30 | 1 |
| 4 | 2020-08-23 03:32:55 | 1 |
+--------+---------------------+---------------+
4 rows in set (0.00 sec)
Expectation: the rows with "1" should show up first.
Actual results: ORDER BY is ignored, and the resultset is sorted by primary key
I have a hunch it has something to do with MySQL storing timestamps as UTC and displaying them in the current timezone. My current timezone is GMT+8. However, it still doesn't make sense -- I am sorting the results based on the aliased expression, and the expression's value is clearly shown in the resultset.
MySQL version 8.0.21.
I also tried moving the expression to the ORDER BY clause, and the results are the same.
I don't know why but it compares wrong timezones in the background and thus values at the end are correct, but comparisons are invalid (for specific timezones).
When you query a TIMESTAMP value, MySQL converts the UTC value back to
your connection’s time zone. Note that this conversion does not take
place for other temporal data types such as DATETIME.
https://www.mysqltutorial.org/mysql-timestamp.aspx/
Changing type from TIMESTAMP to DATETIME fixes problem.
Other solution may be casting to the decimal number.
SELECT b.box_id, updated_at, FORMAT((b.updated_at < NOW() - INTERVAL 3 HOUR),0) AS more_than_3hr
FROM boxes b
ORDER BY more_than_3hr DESC;
From the documentation:
https://dev.mysql.com/doc/refman/8.0/en/user-variables.html
HAVING, GROUP BY, and ORDER BY, when referring to a variable that is assigned a value in the select expression list do not work as expected because the expression is evaluated on the client and thus can use stale column values from a previous row.
Basically, you can't use a variable name you created with "AS" in your sorting.
The solution is to use the verbose statement you used for the AS in sorting. Yeah, it's verbose. 🤷♂️ It is what it is.
I have a column in my database called time. the type of this column is timestamp and Default value is CURRENT_TIMESTAMP
But after some inserts, in phpMyAdmin it shows the value as datetime, e.g. 2019-05-05 04:24:45 and even the Timezone is shown there and can be changed!
I thought MySQL's timestamp is 4 bytes (compared to 8 bytes of datetime) and doesn't store timezone and data is same as INT(10) such as: 1557094115 (seconds passed since 1970 or something like that)
Can any one please explain this, is it a bug or something?
MySQL version 5.7.25
Edit 1 (Screenshots):
It is a TIMESTAMP column, with default value of CURRENT_TIMESTAMP
As you see it is shown as DATETIME and I cannot compare it with integer value of unix_timestamp... also we can change TimeZone to any value (I thought timestamp doesn't store timezone...)
Edit 2:
If (based on one answer) MySQL stores it as an integer internally, then why can't I compare it with integers? (the following query won't work)
DELETE FROM `table` WHERE time < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL :days DAY))
SQLSTATE[22007]: Invalid datetime format: 1292 Incorrect datetime value: '1555980012' for column 'time' at row 1
I also tried it in Sequel Pro and MySQLWorkbench with same results
If you need to see 1557094115, then apply the function UNIX_TIMESTAMP() to the TIMESTAMP or DATETIME column. It's inverse is FROM_UNIXTIME().
mysql> SELECT UNIX_TIMESTAMP("2019-05-05 04:24:45"), FROM_UNIXTIME(1557055485);
+---------------------------------------+---------------------------+
| UNIX_TIMESTAMP("2019-05-05 04:24:45") | FROM_UNIXTIME(1557055485) |
+---------------------------------------+---------------------------+
| 1557055485 | 2019-05-05 04:24:45 |
+---------------------------------------+---------------------------+
More
The internal storage for TIMESTAMP is 1557055485 in UTC; the timezone is added/removed as it is fetched/stored.
The internal storage for DATETIME is (logically, but not actually) the string "2019-05-05 04:24:45" with no hint of timezone. (Actually, it is packed into 5 bytes in some fashion.)
Without any conversion function, fetching TIMESTAMP and DATETIME look the same:
CREATE TABLE `dtts` (
`ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`dt` datetime DEFAULT NULL,
`just_date` date NOT NULL,
`di` int(11) DEFAULT NULL,
`ts_int` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci
1 row in set (0.00 sec)
mysql> select * from dtts
-> ;
+---------------------+---------------------+------------+------------+------------+
| ts | dt | just_date | di | ts_int |
+---------------------+---------------------+------------+------------+------------+
| 2017-06-26 17:52:53 | 2011-06-08 20:45:55 | 2011-06-08 | 20110608 | 1465404577 |
| 2017-06-26 17:52:53 | 2013-03-10 02:35:47 | 2013-03-10 | 20130310 | 1465404577 |
Adding NOW() to both, then SELECTing:
mysql> INSERT INTO dtts (ts, dt) VALUES (NOW(), NOW());
Query OK, 1 row affected, 1 warning (0.00 sec)
| 2019-05-08 14:14:07 | 2019-05-08 14:14:07 | 0000-00-00 | NULL | NULL |
+---------------------+---------------------+------------+------------+------------+
DateTime doesn't store timezone information (it's value only), while MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and convert from UTC to the current time zone for retrieval. What you are seeing from PhpMyAdmin is the retrieved value, not stored value.
Since MySQL 5.6.4, the storage of DateTime has been improved from 8 bytes to 5 bytes (+ fractional seconds storage) Reference
Since MySQL 5.6.4, the DATETIME field requires 5 bytes + 3 bytes fractional. The TIMESTAMP type requires 4 bytes + 3 bytes fractional. Neither of these data types store time zone information. However, both MySQL and phpMyAdmin display TIMESTAMP fields according to the timezone of the database server. You can retrieve the database server's timezone info with the following statements:
SELECT ##global.time_zone, ##session.time_zone;
SELECT EXTRACT(HOUR FROM (TIMEDIFF(NOW(), UTC_TIMESTAMP))) AS `timezone`
If you would like phpMyAdmin to display a different timezone from the database server, you can set the SessionTimeZone property inside of phpMyAdmin's config.inc.php file.
I have a table:
CREATE TABLE `test` (
`t` varchar(255) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8
and update it by:
update test set t = rand();
then the data in the table is:
+---------------------+
| t |
+---------------------+
| 0.24891147599454175 |
| 0.5710943421079725 |
| 0.10873731328988198 |
+---------------------+
as you see, every 't' is different
when update the table using:
update test set t = now(6);
and every 't' is the same:
+----------------------------+
| t |
+----------------------------+
| 2018-03-13 23:40:40.380817 |
| 2018-03-13 23:40:40.380817 |
| 2018-03-13 23:40:40.380817 |
+----------------------------+
what happends when update a column by rand() or by now()?
RAND() is recalculated per row. While NOW() will allways be the same value anywhere in the same statement. That all has nothing to do with UPDATE. It works the same with SELECT. Try the follwing:
select rand()
from (select 1 union all select 1 union all select 1) x;
select now(6)
from (select 1 union all select 1 union all select 1) x;
You will see the same effect.
Demo: http://rextester.com/JGJWV3591
The now() function returns the time when the statement begin to execute and is not recalled for every row while RAND() is recalled for every row. From the MYSQL documentation:
NOW() returns a constant time that indicates the time at which the
statement began to execute. (Within a stored function or trigger,
NOW() returns the time at which the function or triggering statement
began to execute.) This differs from the behavior for SYSDATE(), which
returns the exact time at which it executes.
so you can use SYSDATE() instead of NOW(). This way every row will be different
update test set t = SYSDATE();
The documentation for MySQL functions rand() and now() explain how those functions work, so I won't repeat much of that here. For reference:
https://dev.mysql.com/doc/refman/5.7/en/mathematical-functions.html#function_rand
https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html#function_now
When you run the first statement with rand(), you get random floating point numbers greater than or equal to 0 and less than 1. When you run the second statement:
update test set t = now(6);
The seconds value is represented with a precision of 6 decimal places, as specified. The overall value returned represents the time at which the statement began to execute. That's why all the values are identical.
I have a database with a created_at column containing the datetime in Y-m-d H:i:s format.
The latest datetime entry is 2011-09-28 00:10:02.
I need the query to be relative to the latest datetime entry.
The first value in the query should be the latest datetime entry.
The second value in the query should be the entry closest to 7 days from the first value.
The third value should be the entry closest to 7 days from the second value.
REPEAT #3.
What I mean by "closest to 7 days from":
The following are dates, the interval I desire is a week, in seconds a week is 604800 seconds.
7 days from the first value is equal to 1316578202 (1317183002-604800)
the value closest to 1316578202 (7 days) is... 1316571974
unix timestamp | Y-m-d H:i:s
1317183002 | 2011-09-28 00:10:02 -> appear in query (first value)
1317101233 | 2011-09-27 01:27:13
1317009182 | 2011-09-25 23:53:02
1316916554 | 2011-09-24 22:09:14
1316836656 | 2011-09-23 23:57:36
1316745220 | 2011-09-22 22:33:40
1316659915 | 2011-09-21 22:51:55
1316571974 | 2011-09-20 22:26:14 -> closest to 7 days from 1317183002 (first value)
1316499187 | 2011-09-20 02:13:07
1316064243 | 2011-09-15 01:24:03
1315967707 | 2011-09-13 22:35:07 -> closest to 7 days from 1316571974 (second value)
1315881414 | 2011-09-12 22:36:54
1315794048 | 2011-09-11 22:20:48
1315715786 | 2011-09-11 00:36:26
1315622142 | 2011-09-09 22:35:42
I would really appreciate any help, I have not been able to do this via mysql and no online resources seem to deal with relative date manipulation such as this. I would like the query to be modular enough to be able to change the interval weekly, monthly, or yearly. Thanks in advance!
Answer #1 Reply:
SELECT
UNIX_TIMESTAMP(created_at)
AS unix_timestamp,
(
SELECT MIN(UNIX_TIMESTAMP(created_at))
FROM my_table
WHERE created_at >=
(
SELECT max(created_at) - 7
FROM my_table
)
)
AS `random_1`,
(
SELECT MIN(UNIX_TIMESTAMP(created_at))
FROM my_table
WHERE created_at >=
(
SELECT MAX(created_at) - 14
FROM my_table
)
)
AS `random_2`
FROM my_table
WHERE created_at =
(
SELECT MAX(created_at)
FROM my_table
)
Returns:
unix_timestamp | random_1 | random_2
1317183002 | 1317183002 | 1317183002
Answer #2 Reply:
RESULT SET:
This is the result set for a yearly interval:
id | created_at | period_index | period_timestamp
267 | 2010-09-27 22:57:05 | 0 | 1317183002
1 | 2009-12-10 15:08:00 | 1 | 1285554786
I desire this result:
id | created_at | period_index | period_timestamp
626 | 2011-09-28 00:10:02 | 0 | 0
267 | 2010-09-27 22:57:05 | 1 | 1317183002
I hope this makes more sense.
It's not exactly what you asked for, but the following example is pretty close....
Example 1:
select
floor(timestampdiff(SECOND, tbl.time, most_recent.time)/604800) as period_index,
unix_timestamp(max(tbl.time)) as period_timestamp
from
tbl
, (select max(time) as time from tbl) most_recent
group by period_index
gives results:
+--------------+------------------+
| period_index | period_timestamp |
+--------------+------------------+
| 0 | 1317183002 |
| 1 | 1316571974 |
| 2 | 1315967707 |
+--------------+------------------+
This breaks the dataset into groups based on "periods", where (in this example) each period is 7-days (604800 seconds) long. The period_timestamp that is returned for each period is the 'latest' (most recent) timestamp that falls within that period.
The period boundaries are all computed based on the most recent timestamp in the database, rather than computing each period's start and end time individually based on the timestamp of the period before it. The difference is subtle - your question requests the latter (iterative approach), but I'm hoping that the former (approach I've described here) will suffice for your needs, since SQL doesn't lend itself well to implementing iterative algorithms.
If you really do need to determine each period based on the timestamp in the previous period, then your best bet is going to be an iterative approach -- either using a programming language of your choice (like php), or by building a stored procedure that uses a cursor.
Edit #1
Here's the table structure for the above example.
CREATE TABLE `tbl` (
`id` int(10) unsigned NOT NULL auto_increment PRIMARY KEY,
`time` datetime NOT NULL
)
Edit #2
Ok, first: I've improved the original example query (see revised "Example 1" above). It still works the same way, and gives the same results, but it's cleaner, more efficient, and easier to understand.
Now... the query above is a group-by query, meaning it shows aggregate results for the "period" groups as I described above - not row-by-row results like a "normal" query. With a group-by query, you're limited to using aggregate columns only. Aggregate columns are those columns that are named in the group by clause, or that are computed by an aggregate function like MAX(time)). It is not possible to extract meaningful values for non-aggregate columns (like id) from within the projection of a group-by query.
Unfortunately, mysql doesn't generate an error when you try to do this. Instead, it just picks a value at random from within the grouped rows, and shows that value for the non-aggregate column in the grouped result. This is what's causing the odd behavior the OP reported when trying to use the code from Example #1.
Fortunately, this problem is fairly easy to solve. Just wrap another query around the group query, to select the row-by-row information you're interested in...
Example 2:
SELECT
entries.id,
entries.time,
periods.idx as period_index,
unix_timestamp(periods.time) as period_timestamp
FROM
tbl entries
JOIN
(select
floor(timestampdiff( SECOND, tbl.time, most_recent.time)/31536000) as idx,
max(tbl.time) as time
from
tbl
, (select max(time) as time from tbl) most_recent
group by idx
) periods
ON entries.time = periods.time
Result:
+-----+---------------------+--------------+------------------+
| id | time | period_index | period_timestamp |
+-----+---------------------+--------------+------------------+
| 598 | 2011-09-28 04:10:02 | 0 | 1317183002 |
| 996 | 2010-09-27 22:57:05 | 1 | 1285628225 |
+-----+---------------------+--------------+------------------+
Notes:
Example 2 uses a period length of 31536000 seconds (365-days). While Example 1 (above) uses a period of 604800 seconds (7-days). Other than that, the inner query in Example 2 is the same as the primary query shown in Example 1.
If a matching period_time belongs to more than one entry (i.e. two or more entries have the exact same time, and that time matches one of the selected period_time values), then the above query (Example 2) will include multiple rows for the given period timestamp (one for each match). Whatever code consumes this result set should be prepared to handle such an edge case.
It's also worth noting that these queries will perform much, much better if you define an index on your datetime column. For my example schema, that would look like this:
ALTER TABLE tbl ADD INDEX idx_time ( time )
If you're willing to go for the closest that is after the week is out then this'll work. You can extend it to work out the closest but it'll look so disgusting it's probably not worth it.
select unix_timestamp
, ( select min(unix_tstamp)
from my_table
where sql_tstamp >= ( select max(sql_tstamp) - 7
from my_table )
)
, ( select min(unix_tstamp)
from my_table
where sql_tstamp >= ( select max(sql_tstamp) - 14
from my_table )
)
from my_table
where sql_tstamp = ( select max(sql_tstamp)
from my_table )
I'm trying to store a 12/24hr (ie; 00:00) clock time in a MySQL database. At the moment I am using the time datatype. This works ok but it insists on adding the seconds to the column. So you enter 09:20 and it is stored as 09:20:00. Is there any way I can limit it in MySQL to just 00:00?
That doesn't look possible. The TIME data type is defined to represent the time of the day (or elapsed time) with a 1 second resolution. However, you can always use the DATE_FORMAT() function to format your field as HH:MM in a SELECT query:
SELECT DATE_FORMAT(NOW(), '%k:%i');
+-----------------------------+
| DATE_FORMAT(NOW(), '%k:%i') |
+-----------------------------+
| 4:09 |
+-----------------------------+
1 row in set (0.00 sec)
SELECT DATE_FORMAT(NOW(), '%H:%i');
+-----------------------------+
| DATE_FORMAT(NOW(), '%H:%i') |
+-----------------------------+
| 04:09 |
+-----------------------------+
1 row in set (0.00 sec)
The TIME column type does not accept any parameter or modifier to define range or precision. You can, however, omit seconds on insert if you are careful:
Be careful about assigning abbreviated values to a TIME column. MySQL
interprets abbreviated TIME values with colons as time of the day.
That is, '11:12' means '11:12:00', not '00:11:12'. MySQL interprets
abbreviated values without colons using the assumption that the two
rightmost digits represent seconds (that is, as elapsed time rather
than as time of day). For example, you might think of '1112' and 1112
as meaning '11:12:00' (12 minutes after 11 o'clock), but MySQL
interprets them as '00:11:12' (11 minutes, 12 seconds). Similarly,
'12' and 12 are interpreted as '00:00:12'.
CREATE TABLE example (
example_id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
elapsed TIME NOT NULL,
PRIMARY KEY (example_id)
);
INSERT INTO example (elapsed) VALUES ('123:45:00'), ('123:45');
SELECT * FROM example;
+------------+-----------+
| example_id | elapsed |
+------------+-----------+
| 1 | 123:45:00 |
| 2 | 123:45:00 |
+------------+-----------+
... and you can remove them on read (if necessary) by applying a proper TIME_FORMAT(), noting that:
If the time value contains an hour part that is greater than 23, the
%H and %k hour format specifiers produce a value larger than the usual
range of 0..23. The other hour format specifiers produce the hour
value modulo 12.
INSERT INTO example (elapsed) VALUES ('2:00');
SELECT example_id, TIME_FORMAT(elapsed, '%k:%i') AS elapsed
FROM example;
+------------+---------+
| example_id | elapsed |
+------------+---------+
| 1 | 123:45 |
| 2 | 123:45 |
| 3 | 2:00 |
+------------+---------+
Since MySQL/5.7.5 you can also use a generated column to get a display value automatically:
-- Completely untested, I don't have 5.7 yet
CREATE TABLE example (
example_id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
elapsed TIME NOT NULL,
-- Size to accomodate for '-838:59:59'
elapsed_display VARCHAR(10) AS (TIME_FORMAT(elapsed, '%k:%i')) VIRTUAL NOT NULL,
PRIMARY KEY (example_id)
);