I am using a mySQL database to store a range of inputs that feed a model. I have a number of different dates that are stored as TIMESTAMP. However, some of the values can be hundreds of years in the future. When I look in the DB, they are stored as '0000-00-00 00:00:00' when the actual timestamp should be something like '2850-12-01 00:00:00'.
While searching on Google, I noticed that the maximum value is sometime in 2038. Has anyone found a work-around for longer-dated TIMESTAMPs?
You can convert them to DATETIME, it will store what you want. Compare:
MariaDB [test]> create table t (t timestamp, d datetime);
Query OK, 0 rows affected (0.59 sec)
MariaDB [test]> insert into t values ('2850-12-01 00:00:00','2850-12-01 00:00:00');
Query OK, 1 row affected, 1 warning (0.08 sec)
MariaDB [test]> select * from t;
+---------------------+---------------------+
| t | d |
+---------------------+---------------------+
| 0000-00-00 00:00:00 | 2850-12-01 00:00:00 |
+---------------------+---------------------+
1 row in set (0.00 sec)
Related
I want to show rows that have updated_at more than 3 hours ago. MySQL seems to be completely ignoring the ORDER BY clause. Any idea why?
Edit: as pointed out by Sebastian, this only occurs in certain timezones, like GMT+5 or GMT+8.
mysql> SET time_zone='+08:00';
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE DATABASE test1; USE test1;
Query OK, 1 row affected (0.01 sec)
Database changed
mysql> CREATE TABLE `boxes` (
-> `box_id` int unsigned NOT NULL AUTO_INCREMENT,
-> `updated_at` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
-> PRIMARY KEY (`box_id`)
-> ) ENGINE=InnoDB;
Query OK, 0 rows affected (0.01 sec)
mysql> INSERT INTO `boxes` (`box_id`, `updated_at`) VALUES
-> (1, '2020-08-22 05:25:35'),
-> (2, '2020-08-26 18:49:05'),
-> (3, '2020-08-23 03:28:30'),
-> (4, '2020-08-23 03:32:55');
Query OK, 4 rows affected (0.00 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> SELECT NOW();
+---------------------+
| NOW() |
+---------------------+
| 2020-08-26 20:49:59 |
+---------------------+
1 row in set (0.00 sec)
mysql> SELECT b.box_id, updated_at, (b.updated_at < NOW() - INTERVAL 3 HOUR) AS more_than_3hr
-> FROM boxes b
-> ORDER BY more_than_3hr DESC;
+--------+---------------------+---------------+
| box_id | updated_at | more_than_3hr |
+--------+---------------------+---------------+
| 1 | 2020-08-22 05:25:35 | 1 |
| 2 | 2020-08-26 18:49:05 | 0 | <--- WHY IS THIS HERE???
| 3 | 2020-08-23 03:28:30 | 1 |
| 4 | 2020-08-23 03:32:55 | 1 |
+--------+---------------------+---------------+
4 rows in set (0.00 sec)
Expectation: the rows with "1" should show up first.
Actual results: ORDER BY is ignored, and the resultset is sorted by primary key
I have a hunch it has something to do with MySQL storing timestamps as UTC and displaying them in the current timezone. My current timezone is GMT+8. However, it still doesn't make sense -- I am sorting the results based on the aliased expression, and the expression's value is clearly shown in the resultset.
MySQL version 8.0.21.
I also tried moving the expression to the ORDER BY clause, and the results are the same.
I don't know why but it compares wrong timezones in the background and thus values at the end are correct, but comparisons are invalid (for specific timezones).
When you query a TIMESTAMP value, MySQL converts the UTC value back to
your connection’s time zone. Note that this conversion does not take
place for other temporal data types such as DATETIME.
https://www.mysqltutorial.org/mysql-timestamp.aspx/
Changing type from TIMESTAMP to DATETIME fixes problem.
Other solution may be casting to the decimal number.
SELECT b.box_id, updated_at, FORMAT((b.updated_at < NOW() - INTERVAL 3 HOUR),0) AS more_than_3hr
FROM boxes b
ORDER BY more_than_3hr DESC;
From the documentation:
https://dev.mysql.com/doc/refman/8.0/en/user-variables.html
HAVING, GROUP BY, and ORDER BY, when referring to a variable that is assigned a value in the select expression list do not work as expected because the expression is evaluated on the client and thus can use stale column values from a previous row.
Basically, you can't use a variable name you created with "AS" in your sorting.
The solution is to use the verbose statement you used for the AS in sorting. Yeah, it's verbose. 🤷♂️ It is what it is.
I have a fairly large and complex mysql query to blackbox test and this query is time sensitive, it has a lot of conditions based on current_timestamp.
My goal is to make some tests so it always passes or fails. I'm thinking of mocking the value of current_timestamp temporarily to a fixed date before running the query and set it back to original value after the query.
Is it something thats doable?
I cannot modify the query itself (i.e.: find replace current_timestamp to something else)
Thanks
By setting system variable 'timestamp'. To restore it to current timestamp again, set it to DEFAULT.
mysql> SET TIMESTAMP = UNIX_TIMESTAMP('2015-01-01');
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT NOW();
+---------------------+
| NOW() |
+---------------------+
| 2015-01-01 00:00:00 |
+---------------------+
1 row in set (0.00 sec)
mysql> SET TIMESTAMP = DEFAULT;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT NOW();
+---------------------+
| NOW() |
+---------------------+
| 2016-03-08 09:50:16 |
+---------------------+
1 row in set (0.00 sec)
I'm using the Play Framework 2.1.2, the JDBC MySQL Connector and Scala 2.10. The following query is my problem:
DB.withConnection { implicit connection =>
SQL("""SELECT SUM(r.dayFrequency)
FROM relationships AS r
WHERE r.id = {id}
AND
(r.date BETWEEN {from} AND {to})""").on(
'id -> id,
'from -> from,
'to -> to).as(scalar[Int](bigDecimalToInt).single)
}
It raises this exception:
Execution exception[[RuntimeException: UnexpectedNullableFound(ColumnName(.SUM(r.dayFrequency),Some(SUM(r.dayFrequency))))]]
The console logs the following query:
SELECT SUM(r.dayFrequency)
FROM relationships AS r
WHERE r.id = 26180
AND
(r.date BETWEEN 2014-08-04 12:00:00.0 AND 2014-08-04 12:00:00.0)
If I run this query on my MySQL Workbench it returns null, which confirms the exception. But with this change in the query it works:
(r.date BETWEEN '2014-08-04' AND '2014-08-04')
For the conversion of Joda DateTime, I use this piece of code: Joda DateTime Field on Play Framework 2.0's Anorm
and frequency and date field looks like the following:
date DATE NOT NULL,
dayFrequency INT
Can anyone help with this problem? Seems that something is wrong with the conversion.
EDIT after first POST below:
From the view I receive date strings like this 2014-08-04 and I convert them into Joda DateTime in my controller to compare them to other and use them in MySQL queries like this:
private def clientDateStringToTimestamp(date: String) = {
val Array(year, month, day) = date.split("-")
new DateTime(year.toInt, month.toInt, day.toInt, 12, 0, 0).getMillis()
}
new DateTime(clientDateStringToTimestamp("2014-08-04"))
For the MySQL queries I want to compare only the date part not the time part.
So i did a simple experiment in mysql:
mysql> create table t (v int);
Query OK, 0 rows affected (0.01 sec)
mysql> insert into t values (null);
Query OK, 1 row affected (0.00 sec)
mysql> select sum(v) from t;
+--------+
| sum(v) |
+--------+
| NULL |
+--------+
1 row in set (0.01 sec)
mysql> insert into t values (1);
Query OK, 1 row affected (0.00 sec)
mysql> select sum(v) from t;
+--------+
| sum(v) |
+--------+
| 1 |
+--------+
1 row in set (0.00 sec)
mysql> update t set v = NULL;
Query OK, 1 row affected (0.00 sec)
Rows matched: 2 Changed: 1 Warnings: 0
mysql> select sum(v) from t;
+--------+
| sum(v) |
+--------+
| NULL |
+--------+
1 row in set (0.00 sec)
So this tells us that a summing nulls to nulls gives us null but summing nulls to numbers gives us numbers.
I suspect that your first query (r.date BETWEEN 2014-08-04 12:00:00.0 AND 2014-08-04 12:00:00.0) returns just rows with null dayFrequency values, where the second query (r.date BETWEEN '2014-08-04' AND '2014-08-04'), which is offset 12 hours earlier returns at least one non-null frequency. So since null is possible, you will have to use scalar[Option[Int]] for the sum, then turn it to 0 with getOrElse. A better way, if you can is to make the dayFrequency column in the database NOT NULL DEFAULT 0. Then it will give you a 0, and you can sum away
Also related, direct support for Joda temporal types in Anorm: https://github.com/playframework/playframework/commit/bdbbbe90822a6fb150c7044e68b33e2e52a7323d
What is the behavior of the TIMESTAMP field type in relation to timezones?
Is any timestamp value inserted to that field inserted as is?
Or does it assume that the timezone of a timestamp value that is inserted is in server local time and converts it UTC?
EDIT:
Here is my test
I ran both PHP date() and MySQL's SELECT NOW() and they are outputting roughly equal timestamps. The results of both is not in UTC time.
I tried inserting to a test table with the value for the TIMESTAMP field by gotten from PHP date()
The value from PHP date() SHOULD have been converted to UTC. However, what I see in the database is not UTC. The value for the TIMESTAMP field is inserted as is.
TIMESTAMP value is always saved as UTC.
MySQL converts TIMESTAMP from current timezone to UTC for storage and back from UTC to the current time zone for retrieval.
The default timezone will be the server timezone and can be set on a connection. See this.
For more details see MySQL Doc
I can explain this through an example. Please execute the queries in mysql console:
mysql> CREATE TABLE `testtable` (
`date_timestamp` TIMESTAMP NOT NULL,
`date_datetime` DATETIME NOT NULL
)
ENGINE = InnoDB;
Query OK, 0 rows affected (0.06 sec)
mysql> SET time_zone = '+00:00';
Query OK, 0 rows affected (0.00 sec)
mysql> insert into testtable values(now(),now());
Query OK, 1 row affected (0.03 sec)
mysql> select * from testtable;
+---------------------+---------------------+
| date_timestamp | date_datetime |
+---------------------+---------------------+
| 2012-10-19 05:01:38 | 2012-10-19 05:01:38 |
+---------------------+---------------------+
1 row in set (0.00 sec)
mysql> SET time_zone = '+05:30';
Query OK, 0 rows affected (0.00 sec)
mysql>
mysql> insert into testtable values(now(),now());
Query OK, 1 row affected (0.03 sec)
mysql> select * from testtable;
+---------------------+---------------------+
| date_timestamp | date_datetime |
+---------------------+---------------------+
| 2012-10-19 10:31:38 | 2012-10-19 05:01:38 |
| 2012-10-19 10:31:47 | 2012-10-19 10:31:47 |
+---------------------+---------------------+
2 rows in set (0.00 sec)
The TIMESTAMPT value is inserted AS IS.
Table field structure:
id fdate tdate name
1 2012-07-07 2012-07-30
2 2012-08-07 2012-09-30
3 2012-07-17 2012-08-30
Here
fdate--->one of the field in table
tdate--->another field in table
I Will pass Two values like 2012-07-07 and 2012-09-30
And I need the output From the above Range:
You Have to Check the condition like the following fdate between2012-07-07and2012-09-30and tdate between2012-07-07and2012-09-30 like that
Expected Output:
month Days
07 43
08 53
09 30
Edit: Based on the authors comments the question is - How do I count all the days and show a total grouped by month when passing it a date range and comparing it to the data in my table.
I know that this doesn't give you the exact result that you want, but I think it will help you in the right direction at the very least:
mysql> create table dateTest (id int(2), fdate date, tdate date);
Query OK, 0 rows affected (0.04 sec)
mysql> insert into dateTest values(1, '2012-07-07', '2012-07-30');
Query OK, 1 row affected (0.00 sec)
mysql> insert into dateTest values(1, '2012-08-07', '2012-09-30');
Query OK, 1 row affected (0.00 sec)
mysql> insert into dateTest values(1, '2012-07-17', '2012-08-30');
Query OK, 1 row affected (0.00 sec)
mysql> select month(tdate) as month, datediff(tdate, fdate) as tally from dateTest group by month(fdate), month(tdate);
+-------+-------+
| month | tally |
+-------+-------+
| 7 | 23 |
| 8 | 44 |
| 9 | 54 |
+-------+-------+
3 rows in set (0.00 sec)
As you can see, it is skipping the middle month as I am grouping by fdate, but it is a step in the right direction for you.
Have a look at the TIMESTAMPDIFF() function in MySQL.
What this allows you to do is pass in two TIMESTAMP or DATETIME values (or even DATE as MySQL will auto-convert) as well as the unit of time you want to base your difference on.
You can specify MONTH as the unit in the first parameter:
mysql>SELECT TIMESTAMPDIFF(MONTH, '2012-05-05', '2012-06-04')
-- Outputs: 1
mysql> SELECT TIMESTAMPDIFF(YEAR,'2002-05-01','2001-01-01');
-> -1
mysql> SELECT TIMESTAMPDIFF(MINUTE,'2003-02-01','2003-05-01 12:05:55');
-> 128885
Your question is not clear though. But I have this for you.
You need to fetch your data from MySQL table using php. Then you may calculate date difference there.
You may refer to this stackoverflow question How to calculate the difference between two dates using PHP? and the official php documentation about date_diff at http://php.net/manual/en/function.date-diff.php
You can use the between clause in my sql:
select * from tbl where datetime between '2012-07-07' and '2012-09-30';
as an example