MySQL Subquery Truncating Result with NULLIF - mysql

I have the following query:
SELECT
NULLIF(MAX(t.date),'2019-01-15') AS ended
FROM
totals t
This query correctly outputs a date:
> 2019-01-01
But if I reference this query inside of a subquery like so:
SELECT * FROM
(SELECT
NULLIF(MAX(t.date),'2019-01-15') AS ended
FROM
totals t) AS a
This version incorrectly produces a truncated result:
> 201
Can someone help me to understand this behavior and how best to work around it?
Additional Notes:
I am running MySQL version: "5.7.25 MySQL Community Server"
For anyone wanting to test this out, here is an example of a simple test table that is affected by this problem:
CREATE TABLE `totals` (
`date` date NOT NULL,
`value` decimal(10,0) DEFAULT NULL,
PRIMARY KEY (`date`)
);
INSERT INTO `totals` VALUES ('2018-01-01',2000000),('2019-01-01',3000000);

The issue here appears to be a subtle converting/casting issue happening within NULLIF. First, here is a version of your query which does in fact work as expected:
SELECT *
FROM
(
SELECT NULLIF(MAX(t.date), STR_TO_DATE('2019-01-15', '%Y-%m-%d') AS ended
FROM totals t
) AS a
What is happening with your current query is that MySQL is converting the call to MAX(t.date) to text, to match the text literal '2015-01-15'. By ensuring that both arguments to NULLIF are date type, you get the behavior you want.
As for why we are seeing 201 as the string result from the call to NULLIF, I don't have an explanation. But, I can cite the documentation for NULLIF here:
Returns NULL if expr1 = expr2 is true, otherwise returns expr1. This is the same as CASE WHEN expr1 = expr2 THEN NULL ELSE expr1 END.
It is a general rule in SQL that both the if and else branches of a CASE expression should always have the same type. Actually, if we violate this rule, in most databases the CASE expression won't even compile. Applying this to NULLIF means that we should always make sure that both arguments have the same type. Breaking from this rule might run on MySQL (similar to doing a non ANSI compliant GROUP BY with full mode turned off), but it is not something we should choose if we can avoid it.

My guess is that there's implicit datatype conversions happening.
As a workaround, I would take the return from the NULLIF function and convert/cast it back to DATE datatype. Simplest would be to wrap it in DATE() function.
SELECT
DATE( NULLIF(MAX(t.date),'2019-01-15') ) AS ended
^^^^^ ^
We could also try converting the string literal, convert that to a DATE, and see if that fixes the problem:
SELECT
NULLIF(MAX(t.date), DATE('2019-01-15') ) AS ended
^^^^^ ^
Or we can do both:
SELECT
DATE( NULLIF(MAX(t.date), DATE('2019-01-15') ) ) AS ended
^^^^^ ^
^^^^^ ^
There are other expressions we can use for the datatype conversion such as CAST(), CONVERT(), or STR_TO_DATE().
Or we could just use the simple + INTERVAL 0 DAY trick. e.g.
SELECT
NULLIF(MAX(t.date),'2019-01-15' + INTERVAL 0 DAY ) + INTERVAL 0 DAY AS ended
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^

Related

What causes a Date to be <null> while specifically selecting the NOT NULL fields?

In one of my MYSQL databases, I have done the following query :
SELECT * FROM mytable WHERE date_column < 1971-01-01 AND date_column IS NOT NULL;
This returns about 3000 items where the date is shown as NULL in the mysql command line prompt, and in the IntelliJ IDEA mysql database editor.
The column has a DATE type.
The database is a copy of a prod database environment that has been running for a couple years.
How can a record end up in that state ? What is that state ? How can I fix it properly ?
PS: The < 1971/01/01 clause was just to filter the correct dates out of the query. If I just query on the IS NOT NULL clause, I still get these weird dates of course.
I am surprised this works. First, date constants should be surrounded by single quotes:
SELECT *
FROM mytable
WHERE date_column < '1971-01-01' AND date_column IS NOT NULL;
Second, this cannot return a NULL value for date_column. This suggests one of two things:
date_col is actually another type, such as varchar and you are seeing 'NULL' the string, not NULL the value.
another column is NULL.

MySQL comparing 2 DATETIME values of unknown type

It sounds simple but I'm stuck. What I want is to be able to compare two MySQL DATETIME values but due to the open ended nature of how the queries are formed I do not know the datatype of value 1 or value 2. For instance, each value can either be a string that is input by an end user, a date field or a DATETIME field.
Example:
dateTime1 > 1/18/2017 2:30pm
The issue I'm running into is STR_TO_DATE() expects a string and returns null with DATETIME, DATE_FORMAT() expects a date and returns null with a string. I need a function or nested group of functions that will give me the same result regardless of the value of the datatype & would like to address directly in mysql rather than pre-processing or making the user input validation stricter. I used to use CAST(value, DATETIME), however CAST() doesn't read the date correctly in the more recent versions of MySQL (it ignores the am/pm specification). Any ideas?
GREATEST() did not work for me as it would always return null if any component was null, however it led me to COALESCE() which provides me with the solution I am looking for as it returns the first non NULL value. It makes the assumption the value will always be DATE, DATETIME, or a date time string which is the case for my issue:
SELECT COALESCE(STR_TO_DATE(value1, '%c/%e/%Y %r'), value1) > COALESCE(STR_TO_DATE(value2, '%c/%e/%Y %r'), value2)
Note: value1 and value2 are either DATE / DATETIME columns or date time string values
You can consider using the coalesce() function, with a list of different formats used with str_to_date(). coalesce() will ignore any null from the list (but at least one of them should be non-null, or you still get a null).
select
coalesce(
str_to_date('13/18/2017 2:30pm', '%m/%e/%Y %l:%i%p') -- will get a null
, str_to_date('18/13/2017 2:30pm', '%e/%m/%Y %l:%i%p') -- will get a null
, str_to_date('18/01/2017 14:30', '%e/%m/%Y %k:%i')
) as answer;
The code may be brittle, because there are so many different date/time formats that a human user may input.
You may also need to be very familiar with all the different format characters that str_to_date() take. See this manual page for more details:
https://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_date-format
Updated
An earlier version of this answer incorrectly suggested the use of the greatest() function.

UNIX_TIMESTAMP getting null

This is getting NULL, why?
select unix_timestamp(addtime(date(now()), time(from_unixtime(1426705199))))
For some reason, it is running correctly, however it only outputs NULL
I often find, when confronted with an error, that the manual can be a useful resource:
ADDTIME() adds expr2 to expr1 and returns the result. expr1 is a time or datetime expression, and expr2 is a time expression.
You are using a date value for expr1. Try casting it back to datetime expression:
select unix_timestamp(addtime(
timestamp(date(now()))
, time(from_unixtime(1426705199))))
This should work?
SELECT UNIX_TIMESTAMP(NOW())
If you want to get it from your own field then use this..
SELECT UNIX_TIMESTAMP(yourField);

Select 2 or more values in the IF statement

I want to implement following logic in my SQL query:
If some date is not set or it's year is 1970 then select real_date flag to null AND change date to current data, else - real_date is true and no need to change date.
real_date is not an actual field in the table but a flag I need to set up.
I can easily do this using 2 lines in my SELECT section:
, IF (actualDate is NULL OR YEAR(actualDate) = 1970, CURRENT_TIMESTAMP(), actualDate) as actual_date
, IF (actualDate is NULL OR YEAR(actualDate) = 1970, null, true) as real_date
And the question - is it the only way to do it? Don't really like the fact that I had to copy condition again. Can somehow the second select be moved to the first one?
Update: I need to select both actualDate (it will be either fixed to current time version or the actual stored value) and real_date (which will be true or false/null). Maybe I am missing something, but how could COALESCE function help here?
Update 2: Thanks again everyone. Learnt other ways to write it but all of them have check in 2 places. My idea was to have logical condition in only one place but doesn't seem to be possible.
Your variant is pretty and comfortable for supporting code in future, do not change anything.
You can use the COALESCE function.
Returns the first non-NULL value in the list, or NULL if there are no non-NULL values.
You can also use CASE WHEN ... THEN ... END (with multiple WHEN) instead of the two IF
`
Check this here : http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html
But the COALESCE function can do the job for this case.
I might try something like this instead:
…
, COALESCE(NULLIF(actualDate, '1970-01-01'), CURRENT_TIMESTAMP()) AS actual_date
, NULLIF(actualDate <> '1970-01-01', false) AS is_real_date
…
Or maybe the second NULLIF() is not really necessary? Consider this:
…
, COALESCE(NULLIF(actualDate, '1970-01-01'), CURRENT_TIMESTAMP()) AS actual_date
, (actualDate <> '1970-01-01') AS is_real_date
…
The second column could then be true, false or NULL, and you could treat NULL same as false.
This is the most compact form I can think of using COALESCE and without repeating any comparison:
select id, type, real_date, if(real_date, actual_date, now()) as actual_date
from (
select *, coalesce(year(actual_date), 1970) != 1970 as real_date from table1
) as subResult
Working example

Find invalid dates in SQL Server 2008

I have a 300.000 rows table; one of the columns is a varchar() but it really contains a date xx/xx/xxxx or x/x/xxxx or similar. But executing the following test yields an error:
SELECT CAST(MyDate as Datetime) FROM MyTable
The problem is that it doesn’t tell me in which row…
I have executed a series of “manual” updates by trial an error and performed simple updates to fix those, but there’s got to be some weird values that need to either be deleted or fixed.
For example I performed a simple test that fixed about 40 rows:
UPDATE MyTable SET MyDate = REPLACE(MyDate, '/000','/200') FROM MyTable WHERE MyDate like ('%/000%’)
UPDATE MyTable SET MyDate = REPLACE(MyDate, '/190','/199') FROM MyTable WHERE MyDate like ('%/190%’)
This fixed quite a few weird rows that had dates like 01/01/0003 and such. (Dates range from 1998 to 2010).
However, I’d like to know which rows are failing in the above select.
What would be the best way to print those so I can either delete them, edit them or see what to do? Thanks.
SELECT
*
FROM
MyTable
WHERE
ISDATE(MyDate) = 0
From the ISDATE(Transact-SQL) documentation ISDATE(...)
Returns 1 if the expression is a valid datetime value; otherwise, 0.
ISDATE returns 0 if the expression is a datetime2 value.
Did you try the ISDATE function?
Careful with IsDate. I have 1 bad record in a table of thousands. It says 8201-11-30 is a valid date. IsDate should have a YEAR limitation.
ISDATE doesn't seem to be working always.
the following returns 1 in SQL server 2008 R2
Select ISDATE('04- December 20')
But trying to cast the same value to date will fail.
Select cast('04- December 20' as date)
Conversion failed when converting date and/or time from character string.