Setting NULL datetime with Rose::DB::Object and MySQL - mysql

I could be wrong here, but it looks like there's conflicting standards here.
MySQL treats a stored datetime of "0000-00-00 00:00:00" as being equivalent to NULL.
(update - only, it seems, if the datetime is defined as NOT NULL)
But Rose::DB::Object uses DateTime for MySQL DATETIME fields, and trying to set a null DATETIME from "0000-00-00" throws an exception in the DateTime module. ie, I can't create a DateTime object with year 0, month 0, day 0, because this throws an exception in the DateTime module.
I checked in Rose::DB::Object::Metadata::Column::Datetime, and can't see a way of explicitly handling a NULL DateTime when creating an entry or when retrieving.
Am I missing something?
ie, can Rose::DB::Object handle NULL datetime (MySQL) fields even though DateTime (Perl module) can't.
Sample code:
#!/usr/bin/perl
use strict;
use warnings;
use lib 'lib';
use RoseDB::dt_test;
my $dt_entry = RoseDB::dt_test->new();
$dt_entry->date_time_field('0000-00-00');
$dt_entry->save;
1;
__END__
# definition of table as stored in DB
mysql> show create table dt_test \G
*************************** 1. row ***************************
Table: dt_test
Create Table: CREATE TABLE `dt_test` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`date_time_field` datetime DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
with the RoseDB::dt_test module being:
package RoseDB::dt_test;
use strict;
use warnings;
# this module builds up our DB connection and initializes the connection through:
# __PACKAGE__->register_db
use RoseDB;
use base qw(Rose::DB::Object);
__PACKAGE__->meta->setup (
table => 'dt_test',
columns =>
[
id => { type => 'int', primary_key => 1 },
date_time_field => { type => 'datetime' },
],
);
sub init_db { RoseDB->get_dbh }
1;
When I run it, I get the error "Invalid datetime: '0000-00-00' at tmp.pl line 8"
When I change the date to "2010-01-01", it works as expected:
mysql> select * from dt_test\G
*************************** 1. row ***************************
id: 1
date_time_field: 2010-01-01 00:00:00
I finally managed to recreate the NULL MySQL query example!
mysql> create table dt_test(dt_test_field datetime not null);
Query OK, 0 rows affected (0.05 sec)
mysql> insert into dt_test values(null);
ERROR 1048 (23000): Column 'dt_test_field' cannot be null
mysql> insert into dt_test values('0000-00-00');
Query OK, 1 row affected (0.00 sec)
mysql> select * from dt_test;
+---------------------+
| dt_test_field |
+---------------------+
| 0000-00-00 00:00:00 |
+---------------------+
1 row in set (0.00 sec)
mysql> select * from dt_test where dt_test_field is null;
+---------------------+
| dt_test_field |
+---------------------+
| 0000-00-00 00:00:00 |
+---------------------+
1 row in set (0.00 sec)
Looks like the table definitions where the datetimes are defined with "NOT NULL" and then trying to use the MySQL "fake null" is the issue. I'm too tired to play with this now, but I'll see what happens when I change the table structure in the morning.

You should be able to set a datetime column to the literal desired 0000-00-00 00:00:00 value and save it to the database:
$o->mycolumn('0000-00-00 00:00:00');
$o->save;
Such "all zero" values will not be converted to DateTime objects by Rose::DB::Object, but rather will remain as literal strings. There is no semantic DateTime object equivalent for MySQL's 0000-00-00 00:00:00 datetime strings.
Note: 0000-00-00 is a valid date value, but a datetime (or timestamp) value must include the time: 0000-00-00 00:00:00
To set a column to null, pass undef as the column value:
$o->mycolumn(undef);
Of course, if the column definition in the database includes a NOT NULL constraint, the save() won't work.

MySQL allows date types to be incomplete (lacking year, month, and or day). '0000-00-00' is a valid, non-NULL MySQL date. Why do you think it matches IS NULL?
$ echo "select date('0000-00-00') is null" | mysql
date('0000-00-00') is null
0
For comparison:
$ echo "select date('0000-00-32') is null" | mysql
date('0000-00-32') is null
1

Related

Why is MySQL ignoring the ORDER BY on this condition?

I want to show rows that have updated_at more than 3 hours ago. MySQL seems to be completely ignoring the ORDER BY clause. Any idea why?
Edit: as pointed out by Sebastian, this only occurs in certain timezones, like GMT+5 or GMT+8.
mysql> SET time_zone='+08:00';
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE DATABASE test1; USE test1;
Query OK, 1 row affected (0.01 sec)
Database changed
mysql> CREATE TABLE `boxes` (
-> `box_id` int unsigned NOT NULL AUTO_INCREMENT,
-> `updated_at` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
-> PRIMARY KEY (`box_id`)
-> ) ENGINE=InnoDB;
Query OK, 0 rows affected (0.01 sec)
mysql> INSERT INTO `boxes` (`box_id`, `updated_at`) VALUES
-> (1, '2020-08-22 05:25:35'),
-> (2, '2020-08-26 18:49:05'),
-> (3, '2020-08-23 03:28:30'),
-> (4, '2020-08-23 03:32:55');
Query OK, 4 rows affected (0.00 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> SELECT NOW();
+---------------------+
| NOW() |
+---------------------+
| 2020-08-26 20:49:59 |
+---------------------+
1 row in set (0.00 sec)
mysql> SELECT b.box_id, updated_at, (b.updated_at < NOW() - INTERVAL 3 HOUR) AS more_than_3hr
-> FROM boxes b
-> ORDER BY more_than_3hr DESC;
+--------+---------------------+---------------+
| box_id | updated_at | more_than_3hr |
+--------+---------------------+---------------+
| 1 | 2020-08-22 05:25:35 | 1 |
| 2 | 2020-08-26 18:49:05 | 0 | <--- WHY IS THIS HERE???
| 3 | 2020-08-23 03:28:30 | 1 |
| 4 | 2020-08-23 03:32:55 | 1 |
+--------+---------------------+---------------+
4 rows in set (0.00 sec)
Expectation: the rows with "1" should show up first.
Actual results: ORDER BY is ignored, and the resultset is sorted by primary key
I have a hunch it has something to do with MySQL storing timestamps as UTC and displaying them in the current timezone. My current timezone is GMT+8. However, it still doesn't make sense -- I am sorting the results based on the aliased expression, and the expression's value is clearly shown in the resultset.
MySQL version 8.0.21.
I also tried moving the expression to the ORDER BY clause, and the results are the same.
I don't know why but it compares wrong timezones in the background and thus values at the end are correct, but comparisons are invalid (for specific timezones).
When you query a TIMESTAMP value, MySQL converts the UTC value back to
your connection’s time zone. Note that this conversion does not take
place for other temporal data types such as DATETIME.
https://www.mysqltutorial.org/mysql-timestamp.aspx/
Changing type from TIMESTAMP to DATETIME fixes problem.
Other solution may be casting to the decimal number.
SELECT b.box_id, updated_at, FORMAT((b.updated_at < NOW() - INTERVAL 3 HOUR),0) AS more_than_3hr
FROM boxes b
ORDER BY more_than_3hr DESC;
From the documentation:
https://dev.mysql.com/doc/refman/8.0/en/user-variables.html
HAVING, GROUP BY, and ORDER BY, when referring to a variable that is assigned a value in the select expression list do not work as expected because the expression is evaluated on the client and thus can use stale column values from a previous row.
Basically, you can't use a variable name you created with "AS" in your sorting.
The solution is to use the verbose statement you used for the AS in sorting. Yeah, it's verbose. 🤷‍♂️ It is what it is.

MySQL FROM_UNIXTIME() returns null with last_update column of sakila.actor

The actor table in the salika sample schema defines column last_update as a timestamp. I want to render that column using JSON_ARRAY in ISO8601 format. - First shouldn't that be the default rendering for JSON_ARRAY.
From reading the documentation and comments on this website and others it appears that the answer is to use FROM_UNXTIME with an output mask that generates ISO8601 Format.
Unfortunately FROM_UNIXTIME() appears to always return NULL on my database
mysql> select current_timestamp();
+---------------------+
| current_timestamp() |
+---------------------+
| 2018-10-03 17:15:03 |
+---------------------+
1 row in set (0.00 sec)
mysql> select from_unixTime(current_timestamp())
-> ;
+------------------------------------+
| from_unixTime(current_timestamp()) |
+------------------------------------+
| NULL |
+------------------------------------+
1 row in set (0.00 sec)
mysql>
I suspect this may be caused by the fact that I have not installed the timezone configuration files.. However when I try that I get...
mysql -u root -p****** sys <timezone_posix.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1146 (42S02) at line 1: Table 'sys.time_zone' doesn't exist
Sure I've missed something obvious here....
Just try this:
SELECT FROM_UNIXTIME(UNIX_TIMESTAMP(CURRENT_TIMESTAMP()));
As mysql doc says, function FROM_UNIXTIME(unix_timestamp[,format]) only accept parameter as a UNIX_TIMESTAMP, but not a TIMESTAMP.
Luckily, there's a function UNIX_TIMESTAMP(date) transforming various types such as DATE, TIMESTAMP and so on into UNIX_TIMESTAMP. So call UNIX_TIMESTAMP(date) first, and then `FROM_UNIXTIME(
This worked for me in the end...
select json_array("actor_id","first_name","last_name",DATE_FORMAT(convert_tz("last_update", ##session.time_zone, '+00:00'),'%Y-%m-%dT%T%fZ')) "json" from "sakila"."actor"
/
which gives
[
1,
"PENELOPE",
"GUINESS",
"2006-02-15T12:34:33000000Z"
],
[
2,
"NICK",
"WAHLBERG",
"2006-02-15T12:34:33000000Z"
]

MySQL return extra records when using a long type number to filter varchar type

A simple table:
CREATE TABLE `tbl_type_test` (
`uid` varchar(31) NOT NULL DEFAULT '0',
`value` varchar(15) NOT NULL DEFAULT '',
PRIMARY KEY (`uid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
and some records:
'6011656821812318924', 'a'
'6011656821812318925', 'b'
'6011656821812318926', 'c'
when I execute the following SQL, 3 records will return
select * from tbl_type_test where uid = 6011656821812318924;
and this will return 3 records, too. Weird.
select * from tbl_type_test where uid = 6011656821812318900;
if I change the number to string type, as expected, only 1 record will return:
select * from tbl_type_test where uid = '6011656821812318924';
I think the number type and length in the query is the reason, but I don't known the exact.
Any comment will be greatly appreciated.
In all other cases, the arguments are compared as floating-point (real) numbers. - https://dev.mysql.com/doc/refman/5.7/en/type-conversion.html
for example
drop procedure if exists p;
delimiter $$
create procedure p (inval float, inval2 float, inval3 float)
select inval,inval2,inval3;
call p(6011656821812318924,6011656821812318925,6011656821812318926);
+------------+------------+------------+
| inval | inval2 | inval3 |
+------------+------------+------------+
| 6.01166e18 | 6.01166e18 | 6.01166e18 |
+------------+------------+------------+
1 row in set (0.00 sec)
MySQL by default treats 1 and '1' the same however you can change that by setting the MySQL behavior to Strict mode.
set ##GLOBAL.sql_mode = "STRICT_ALL_TABLES";
set ##SESSION.sql_mode = "STRICT_ALL_TABLES";
or you can set these variables in your my.cnf file to be permanent in sql_mode = ''. This way MySQL will throw an error if an incorrect type is used. Read
https://dev.mysql.com/doc/refman/5.7/en/constraint-invalid-data.html
Regards

Why isn't zero date when compared to null, null?

SELECT NULL; prints
<null>
SELECT CAST('0000-00-00' AS DATE); prints
<null>
SELECT NULL IS NULL; prints
1
SELECT CAST('0000-00-00' AS DATE) IS NULL; prints
0
Why isn't the result of the 4th statement 1?
Tested using DataGrip while connecting to MySQL 5.7.17 on windows.
Because the 2. isn't true:
SELECT CAST('0000-00-00' AS DATE); prints
0000-00-00
It doesn't print
NULL
4th statement show the result 0000-00-00
In mysql Only null is null. Everything else is not, so that it returns 1
Thanks to kiks73, iamsankalp89 and moscas for their answers (kiks73's answer - iamsankalp89's answer - moscas's answer) which helped me find the solution/ explanation for the odd behavior that I've encountered.
The reason for the odd behavior was how JDBC driver which is used by DataGrip would handle a zero date.
It would receive the date as zeros from the database then convert it to null.
That would result in DataGrip reading zero dates as nulls because of the JDBC conversion.
Howevenr, because its done at the JDBC layer (not the database layer) the database doesn't see those values as nulls.
To test for that, i ran the database connection and the query through mysql shell, which would print zeros instead of nulls.
Tested in Ubuntu mysql:
SELECT CAST('0000-00-00' AS DATE) ;
The value of the above query is null,
mysql> SELECT CAST('0000-00-00' AS DATE);
+----------------------------+
| CAST('0000-00-00' AS DATE) |
+----------------------------+
| NULL |
+----------------------------+
1 row in set, 1 warning (0.00 sec)
SELECT CAST('0000-00-00' AS DATE) IS NULL;
In the above query, you are trying to check the value is null using IS NULL function and the answer is True so only it's displaying the value as 1.
mysql> SELECT CAST('0000-00-00' AS DATE) IS NULL;
+------------------------------------+
| CAST('0000-00-00' AS DATE) IS NULL |
+------------------------------------+
| 1 |
+------------------------------------+
1 row in set, 1 warning (0.00 sec)
While testing using others,
SELECT CAST('0000-00-00' AS DATE) ;
The value of the above query won't display as NULL,
SELECT CAST('0000-00-00' AS DATE) IS NULL;
The result of above query is False, so only it displays as 0

Mysql 4 vs Mysql 5 auto-increment field on insert

I've learned this along the way but can't figure out where I read it or heard it, as there is nothing I have found online supporting it, but I remember that when upgrading from mysql4.x to mysql5.x, one of the required changes was that the auto-increment field for inserts had to change from '' to NULL if it was included.
I know its not required to have in the insert anyway, but just for point of interest...
Mysql 4.x would allow:
INSERT INTO TABLE (table_id, name, location) VALUES ('', 'john', 'NY');
But mysql 5.x had to have:
INSERT INTO TABLE (table_id, name, location) VALUES (NULL, 'john', 'NY');
I can't find any information on mysql's site to support this, but I know for a fact it throws an error in mysql 5.x and know it worked with '' in 4.x, but where is this documented?
Both the 4.1 and 5.0 docs state that 0 or NULL is required:
No value was specified for the
AUTO_INCREMENT column, so MySQL
assigned sequence numbers
automatically. You can also explicitly
assign NULL or 0 to the column to
generate sequence numbers.
It does not matter, mysql internally still convert to integer
mysql> CREATE TABLE some_test ( id int(10) unsigned NOT NULL auto_increment, primary key(id));
Query OK, 0 rows affected (0.00 sec)
mysql> insert into some_test values ('');
Query OK, 1 row affected, 1 warning (0.00 sec)
mysql> show warnings;
+---------+------+------------------------------------------------------+
| Level | Code | Message |
+---------+------+------------------------------------------------------+
| Warning | 1264 | Out of range value adjusted for column 'id' at row 1 |
+---------+------+------------------------------------------------------+
1 row in set (0.00 sec)
mysql> select * from some_test;
+----+
| id |
+----+
| 1 |
+----+
1 row in set (0.00 sec)
However, I will suggest use 0 to avoid this warning