Confused about comparing date fields with literals - mysql

I am puzzled by the behaviour below. Why does the first SELECT statement return 1 while the second statement returns 0? I expect them both to return 1 as the date is greater than or equal to the literal.
Why does collation affect date comparison? When comparing dates against literals, is it wrong to represent the date (or date time) as a string? If so how should I be doing date vs literal comparisons?
mysql> CREATE DATABASE test;
Query OK, 1 row affected (0.00 sec)
mysql> USE test;
mysql> SET NAMES utf8 COLLATE utf8_general_ci;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE TABLE foo (
bar date NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Query OK, 0 rows affected (0.15 sec)
mysql> INSERT INTO foo (bar) VALUES ('2013-01-01');
Query OK, 1 row affected (0.00 sec)
mysql> SELECT COUNT(*) FROM foo WHERE bar >= '2013-01-01 00:00:00';
+----------+
| COUNT(*) |
+----------+
| 1 |
+----------+
1 row in set (0.00 sec)
mysql> SET NAMES utf8 COLLATE utf8_unicode_ci;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT COUNT(*) FROM foo WHERE bar >= '2013-01-01 00:00:00';
+----------+
| COUNT(*) |
+----------+
| 0 |
+----------+
1 row in set (0.00 sec)

Have you tried
SELECT COUNT(*) FROM foo WHERE bar >= _utf8'2013-01-01 00:00:00'
Explanation's here

You probably assume that the db is doing an IMPLICIT type converion of your literal string into a date and comparing the dates. It looks like the db is doing an IMPLICIT type conversion of the date into a string and comparing the strings. The collation affects this conversion and so affects your result.
Try:
SET NAMES utf8 COLLATE utf8_general_ci;
SELECT * FROM foo;
SET NAMES utf8 COLLATE utf8_unicode_ci;
SELECT * FROM foo;
The two queries should give different results that explain the behaviour.
In any case cha's suggestion will work because you are telling the db to EXPLICITLY convert the literal string to a date and then compare the dates.

Related

What mean by char(40)?

I have a mysql table which has a data structure as follows,
create table data(
....
name char(40) NULL,
...
)
But I could insert names which has characters more than 40 in to name field. Can someone explain what is the actual meaning of char(40)?
You cannot insert a string of more than 40 characters in a column defined with the type CHAR(40).
If you run MySQL in strict mode, you will get an error if you try to insert a longer string.
mysql> create table mytable ( c char(40) );
Query OK, 0 rows affected (0.01 sec)
mysql> insert into mytable (c) values ('Now is the time for all good men to come to the aid of their country.');
ERROR 1406 (22001): Data too long for column 'c' at row 1
If you run MySQL in non-strict mode, the insert will succeed, but only the first 40 characters of your string is stored in the column. The characters beyond 40 are lost, and you get no error.
mysql> set sql_mode='';
Query OK, 0 rows affected (0.00 sec)
mysql> insert into mytable (c) values ('Now is the time for all good men to come to the aid of their country.');
Query OK, 1 row affected, 1 warning (0.01 sec)
mysql> show warnings;
+---------+------+----------------------------------------+
| Level | Code | Message |
+---------+------+----------------------------------------+
| Warning | 1265 | Data truncated for column 'c' at row 1 |
+---------+------+----------------------------------------+
1 row in set (0.00 sec)
mysql> select c from mytable;
+------------------------------------------+
| c |
+------------------------------------------+
| Now is the time for all good men to come |
+------------------------------------------+
1 row in set (0.00 sec)
I recommend operating MySQL in strict mode (strict mode is the default since MySQL 5.7). I would prefer to get an error instead of losing data.

mysql compares accented vs unaccented characters as the same?

This does not make sense to me. Can anyone explain it? I think the column values should be different, so
select * from a1 where f1 = f2;
should find no rows. But...
mysql> create table a1 (f1 varchar(63), f2 varchar(63));
Query OK, 0 rows affected (0.00 sec)
mysql> show create table a1 \G
*************************** 1. row ***************************
Table: a1
Create Table: CREATE TABLE `a1` (
`f1` varchar(63) DEFAULT NULL,
`f2` varchar(63) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
1 row in set (0.00 sec)
mysql>
mysql> insert into a1 values ('EFBBBFD187D0B5D0BBD0BED0B2D0B5D0BA', 'EFBBBFD187D0B5D0BBD0BED0B2D0B5CC81D0BA');
Query OK, 1 row affected (0.02 sec)
mysql> update a1 set f1 = unhex(f1);
Query OK, 1 row affected (0.02 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> update a1 set f2 = unhex(f2);
Query OK, 1 row affected (0.02 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select * from a1;
+-------------------+---------------------+
| f1 | f2 |
+-------------------+---------------------+
| человек | челове́к |
+-------------------+---------------------+
1 row in set (0.00 sec)
mysql>
mysql>
mysql> select * from a1 where f1 = f2;
+-------------------+---------------------+
| f1 | f2 |
+-------------------+---------------------+
| человек | челове́к |
+-------------------+---------------------+
1 row in set (0.00 sec)
mysql> select * from a1 where hex(f1) = hex(f2);
Empty set (0.00 sec)
mysql>
The 3 bytes on the beginning, EFBBBF, is "BOM", which indicates that the text is UTF-8-encoded.
The rest look like Cyrillic челове́к, except for the "CC81 -- NSM COMBINING ACUTE ACCENT"
Some collations, including utf8mb4_0900_ai_ci, handle "combining accents", some do not. The "ai" means "accent insensitive".
I would understand this equivalence for a "latin" e. I don't know the rules for a CYRILLIC SMALL LETTER IE, which looks the same е, but is encoded differently.
You might want COLLATE utf8mb4_0900_as_ci, which is "accent sensitive and case insensitive".
Character equivalence is defined by the collation used by the columns in question. A collation defines every pair of characters as equal, less than, or greater than, and this is used for comparisons and for sorting.
Your table uses utf8mb4_0900_ai_ci as the default collation, and this applies to all the columns, since they do not define a collation to override the table's default.
It's pretty common for collations to treat accented characters as equal to their unaccented versions.
If you want to choose a different collation, you may.

MySQL varchar column on group_concat shows as text in View

I have column in base table as acct_num varchar(25) and I am creating a View and GROUP_CONCAT() is set to this column. This is shown as text datatype in show create view. Is it possible to have VARCHAR(25) datatype for GROUP_CONCAT(acct_num) column. Please advise.
What I learnt from MySQL reference:
I though to cast as varchar(25) but CAST can be applied as CHAR not as VARCHAR
There is option to set GLOBAL_SET_GROUP_CONCAT_VALUE = 512 so that you can get output of GROUP_CONCAT() as varchar() - but it didn't work
out for me.
You can set group_concat_max_len to achieve that.
Here is a demo:
SQL:
-- To change the setting globally
set global group_concat_max_len = 512;
-- To change the setting only for current session
set group_concat_max_len = 512;
create table t1(acct_num varchar(25));
create view v1 as select group_concat(acct_num) as gc_acct_num from t1;
desc v1;
Output:
mysql> -- To change the setting globally
mysql> set global group_concat_max_len = 512;
Query OK, 0 rows affected (0.00 sec)
mysql> -- To change the setting only for current session
mysql> set group_concat_max_len = 512;
Query OK, 0 rows affected (0.00 sec)
mysql> create table t1(acct_num varchar(25));
Query OK, 0 rows affected (0.01 sec)
mysql> create view v1 as select group_concat(acct_num) as gc_acct_num from t1;
Query OK, 0 rows affected (0.00 sec)
mysql> desc v1;
+-------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+-------+
| gc_acct_num | varchar(512) | YES | | NULL | |
+-------------+--------------+------+-----+---------+-------+
1 row in set (0.00 sec)

Joda time issue with MySQL statement

I'm using the Play Framework 2.1.2, the JDBC MySQL Connector and Scala 2.10. The following query is my problem:
DB.withConnection { implicit connection =>
SQL("""SELECT SUM(r.dayFrequency)
FROM relationships AS r
WHERE r.id = {id}
AND
(r.date BETWEEN {from} AND {to})""").on(
'id -> id,
'from -> from,
'to -> to).as(scalar[Int](bigDecimalToInt).single)
}
It raises this exception:
Execution exception[[RuntimeException: UnexpectedNullableFound(ColumnName(.SUM(r.dayFrequency),Some(SUM(r.dayFrequency))))]]
The console logs the following query:
SELECT SUM(r.dayFrequency)
FROM relationships AS r
WHERE r.id = 26180
AND
(r.date BETWEEN 2014-08-04 12:00:00.0 AND 2014-08-04 12:00:00.0)
If I run this query on my MySQL Workbench it returns null, which confirms the exception. But with this change in the query it works:
(r.date BETWEEN '2014-08-04' AND '2014-08-04')
For the conversion of Joda DateTime, I use this piece of code: Joda DateTime Field on Play Framework 2.0's Anorm
and frequency and date field looks like the following:
date DATE NOT NULL,
dayFrequency INT
Can anyone help with this problem? Seems that something is wrong with the conversion.
EDIT after first POST below:
From the view I receive date strings like this 2014-08-04 and I convert them into Joda DateTime in my controller to compare them to other and use them in MySQL queries like this:
private def clientDateStringToTimestamp(date: String) = {
val Array(year, month, day) = date.split("-")
new DateTime(year.toInt, month.toInt, day.toInt, 12, 0, 0).getMillis()
}
new DateTime(clientDateStringToTimestamp("2014-08-04"))
For the MySQL queries I want to compare only the date part not the time part.
So i did a simple experiment in mysql:
mysql> create table t (v int);
Query OK, 0 rows affected (0.01 sec)
mysql> insert into t values (null);
Query OK, 1 row affected (0.00 sec)
mysql> select sum(v) from t;
+--------+
| sum(v) |
+--------+
| NULL |
+--------+
1 row in set (0.01 sec)
mysql> insert into t values (1);
Query OK, 1 row affected (0.00 sec)
mysql> select sum(v) from t;
+--------+
| sum(v) |
+--------+
| 1 |
+--------+
1 row in set (0.00 sec)
mysql> update t set v = NULL;
Query OK, 1 row affected (0.00 sec)
Rows matched: 2 Changed: 1 Warnings: 0
mysql> select sum(v) from t;
+--------+
| sum(v) |
+--------+
| NULL |
+--------+
1 row in set (0.00 sec)
So this tells us that a summing nulls to nulls gives us null but summing nulls to numbers gives us numbers.
I suspect that your first query (r.date BETWEEN 2014-08-04 12:00:00.0 AND 2014-08-04 12:00:00.0) returns just rows with null dayFrequency values, where the second query (r.date BETWEEN '2014-08-04' AND '2014-08-04'), which is offset 12 hours earlier returns at least one non-null frequency. So since null is possible, you will have to use scalar[Option[Int]] for the sum, then turn it to 0 with getOrElse. A better way, if you can is to make the dayFrequency column in the database NOT NULL DEFAULT 0. Then it will give you a 0, and you can sum away
Also related, direct support for Joda temporal types in Anorm: https://github.com/playframework/playframework/commit/bdbbbe90822a6fb150c7044e68b33e2e52a7323d

MySQL - TIMESTAMP field type automatically converts to UTC?

What is the behavior of the TIMESTAMP field type in relation to timezones?
Is any timestamp value inserted to that field inserted as is?
Or does it assume that the timezone of a timestamp value that is inserted is in server local time and converts it UTC?
EDIT:
Here is my test
I ran both PHP date() and MySQL's SELECT NOW() and they are outputting roughly equal timestamps. The results of both is not in UTC time.
I tried inserting to a test table with the value for the TIMESTAMP field by gotten from PHP date()
The value from PHP date() SHOULD have been converted to UTC. However, what I see in the database is not UTC. The value for the TIMESTAMP field is inserted as is.
TIMESTAMP value is always saved as UTC.
MySQL converts TIMESTAMP from current timezone to UTC for storage and back from UTC to the current time zone for retrieval.
The default timezone will be the server timezone and can be set on a connection. See this.
For more details see MySQL Doc
I can explain this through an example. Please execute the queries in mysql console:
mysql> CREATE TABLE `testtable` (
`date_timestamp` TIMESTAMP NOT NULL,
`date_datetime` DATETIME NOT NULL
)
ENGINE = InnoDB;
Query OK, 0 rows affected (0.06 sec)
mysql> SET time_zone = '+00:00';
Query OK, 0 rows affected (0.00 sec)
mysql> insert into testtable values(now(),now());
Query OK, 1 row affected (0.03 sec)
mysql> select * from testtable;
+---------------------+---------------------+
| date_timestamp | date_datetime |
+---------------------+---------------------+
| 2012-10-19 05:01:38 | 2012-10-19 05:01:38 |
+---------------------+---------------------+
1 row in set (0.00 sec)
mysql> SET time_zone = '+05:30';
Query OK, 0 rows affected (0.00 sec)
mysql>
mysql> insert into testtable values(now(),now());
Query OK, 1 row affected (0.03 sec)
mysql> select * from testtable;
+---------------------+---------------------+
| date_timestamp | date_datetime |
+---------------------+---------------------+
| 2012-10-19 10:31:38 | 2012-10-19 05:01:38 |
| 2012-10-19 10:31:47 | 2012-10-19 10:31:47 |
+---------------------+---------------------+
2 rows in set (0.00 sec)
The TIMESTAMPT value is inserted AS IS.