I am working with MySQL Community edition 5.1. This is a strange problem. I have a table where a certain column value is null. Lets say we have
table1 is
| col_1 | id |
null 1
When I do this:
select col_1 from table1 where id = 1
I get this ->
| col_1 |
null
But when I do this:
select col_1 from table1 where id = 1 and col_1 != "x";
The query does not return any results.
I am expecting the second query here to return the same as the first query, since the value of col_1 is obviously not "x". Why is this happening? What can I do to the second query so that it works the way it should, ie. return the row with id = 1 and all values of col_1 except for "x"?
Thanks!!
As others have mentioned, comparisons with one or more NULL operands result in NULL. You can instead use MySQL's NULL-safe equality operator, <=>:
SELECT col_1 FROM table1 WHERE id = 1 AND NOT col_1 <=> "x"
See The Behavior of NULLs in SQL
A boolean comparison between two values involving a NULL returns
neither true nor false, but unknown in SQL's three-valued logic. [3]
For example, neither NULL equals NULL nor NULL not-equals NULL is
true. Testing whether a value is NULL requires an expression such as
IS NULL or IS NOT NULL.
So, you could change your query to:
select col_1
from table1
where id = 1
and (col_1 is null or col_1 != 'x');
Comparing NULLs to a value using != in mysql can yield unexpected results. It's a dumb quirk of SQL (sorry I don't have a better answer) NULL to them means "not having a value" so it can't be compared. See http://dev.mysql.com/doc/refman/5.0/en/working-with-null.html
try:
select col_1 from table1 where id = 1 and (col_1 != "x" OR col_1 IS NULL);
Related
based on below statement
select null
union
select null
the output of the above statement is:
null
While this statement :
select null
union all
select null
Outputs:
null
null
As null <> null then how the value null is treated here and in which datatype it is considered
In standard SQL, UNION removes duplicate records, UNION ALL does not. Happlily your RDBMS is clever enough to figure out that NULL IS NULL, and eliminates duplicate rows when UNION is used.
NB : null = null is unknown, however null <> null is unknown as well. The only way to check for nullity is to use something like IS NULL.
SELECT case when null <> null then 1 else 0 end; --> yields : 0
SELECT case when null = null then 1 else 0 end; --> yields : 0
SELECT case when null IS null then 1 else 0 end; --> yields : 1
UNION is a set operator and involves checking for "duplicate rows". The specs define duplicate rows as "not being distinct". An excerpt from SQL-92 specs:
Two values are said to be not distinct if either: both are the null
value, or they compare equal according to Subclause 8.2, "<comparison
predicate>". Otherwise they are distinct. Two rows (or partial rows)
are distinct if at least one of their pairs of respective values is
distinct. Otherwise they are not distinct. The result of evaluating
whether or not two values or two rows are distinct is never unknown.
(emphasis mine). So, in this example:
select null
union all
select null
the two rows are considered duplicates of each other because the null values in first column are considered not distinct... i.e. same. And by definition UNION returns only one row from a set of duplicate rows.
I am considering that you know the difference between UNION (deduplicates results) and UNION ALL
select 'x' from dual where null is null \\results with x
In this case null is actually null. Which means union returns correct result (deduplicated)
I'm having a problem with some SQL queries that I cant figure out.
SELECT * FROM MasterList WHERE Requested <> "Yes";
If I run the above query on my table I get 60 records returned. However There should be close to 300. I think the issue is, some of the records are just blank in the Requested field and others are NULL. But I would have thought NULL would still count as not equal to "Yes" wouldnt it? If not, is there a way around that without having to go back and 'blank' all the null fields?
Nulls are not counted in comparison, if you want null values to be returned then you need to execute the following query:
SELECT * FROM MasterList WHERE Requested <> "Yes" OR Requested IS NULL;
<=>
NULL-safe equal. This operator performs an equality comparison like the = operator, but returns 1 rather than NULL if both operands are NULL, and 0 rather than NULL if one operand is NULL.
mysql> SELECT 1 <=> 1, NULL <=> NULL, 1 <=> NULL;
-> 1, 1, 0
mysql> SELECT 1 = 1, NULL = NULL, 1 = NULL;
-> 1, NULL, NULL
in your case use:
SELECT * FROM MasterList WHERE not Requested <=> "Yes"
It happens because null <> 'Yes' expression evaluates to null, thus you should add a separate is null check to include records that meet this condition in your result set:
select * from MasterList where Requested <> "Yes" or Requested is null
P.S.: comparison of null with anything, even with null itself, always returns null.
I've struggled with the same issue and someone introduced me to the IS DISTINCT FROM construct. Very useful but not available for all DBMS!
SELECT * FROM MasterList WHERE Requested IS DISTINCT FROM "Yes";
Could someone please explain the following behavior in SQL?
SELECT * FROM MyTable WHERE MyColumn != NULL (0 Results)
SELECT * FROM MyTable WHERE MyColumn <> NULL (0 Results)
SELECT * FROM MyTable WHERE MyColumn IS NOT NULL (568 Results)
<> is Standard SQL-92; != is its equivalent. Both evaluate for values, which NULL is not -- NULL is a placeholder to say there is the absence of a value.
Which is why you can only use IS NULL/IS NOT NULL as predicates for such situations.
This behavior is not specific to SQL Server. All standards-compliant SQL dialects work the same way.
Note: To compare if your value is not null, you use IS NOT NULL, while to compare with not null value, you use <> 'YOUR_VALUE'. I can't say if my value equals or not equals to NULL, but I can say if my value is NULL or NOT NULL. I can compare if my value is something other than NULL.
NULL has no value, and so cannot be compared using the scalar value operators.
In other words, no value can ever be equal to (or not equal to) NULL because NULL has no value.
Hence, SQL has special IS NULL and IS NOT NULL predicates for dealing with NULL.
Note that this behavior is the default (ANSI) behavior.
If you:
SET ANSI_NULLS OFF
http://msdn.microsoft.com/en-us/library/ms188048.aspx
You'll get different results.
SET ANSI_NULLS OFF will apparently be going away in the future...
We use
SELECT * FROM MyTable WHERE ISNULL(MyColumn, ' ') = ' ';
to return all rows where MyColumn is NULL or all rows where MyColumn is an empty string. To many an "end user", the NULL vs. empty string issue is a distinction without a need and point of confusion.
In SQL, anything you evaluate / compute with NULL results into UNKNOWN
This is why SELECT * FROM MyTable WHERE MyColumn != NULL or SELECT * FROM MyTable WHERE MyColumn <> NULL gives you 0 results.
To provide a check for NULL values, isNull function is provided.
Moreover, you can use the IS operator as you used in the third query.
The only test for NULL is IS NULL or IS NOT NULL. Testing for equality is nonsensical because by definition one doesn't know what the value is.
Here is a wikipedia article to read:
https://en.wikipedia.org/wiki/Null_(SQL)
NULL Cannot be compared to any value using the comparison operators. NULL = NULL is false. Null is not a value. The IS operator is specially designed to handle NULL comparisons.
null represents no value or an unknown value. It doesn’t specify why there is no value, which can lead to some ambiguity.
Suppose you run a query like this:
SELECT *
FROM orders
WHERE delivered=ordered;
that is, you are looking for rows where the ordered and delivered dates are the same.
What is to be expected when one or both columns are null?
Because at least one of the dates is unknown, you cannot expect to say that the 2 dates are the same. This is also the case when both dates are unknown: how can they be the same if we don’t even know what they are?
For this reason, any expression treating null as a value must fail. In this case, it will not match. This is also the case if you try the following:
SELECT *
FROM orders
WHERE delivered<>ordered;
Again, how can we say that two values are not the same if we don’t know what they are.
SQL has a specific test for missing values:
IS NULL
Specifically it is not comparing values, but rather it seeks out missing values.
Finally, as regards the != operator, as far as I am aware, it is not actually in any of the standards, but it is very widely supported. It was added to make programmers from some languages feel more at home. Frankly, if a programmer has difficulty remembering what language they’re using, they’re off to a bad start.
I would like to suggest this code I made to find if there is a change in a value,
i being the new value and d being the old (although the order does not matter). For that matter, a change from value to null or vice versa is a change but from null to null is not (of course, from value to another value is a change but from value to the same it is not).
CREATE FUNCTION [dbo].[ufn_equal_with_nulls]
(
#i sql_variant,
#d sql_variant
)
RETURNS bit
AS
BEGIN
DECLARE #in bit = 0, #dn bit = 0
if #i is null set #in = 1
if #d is null set #dn = 1
if #in <> #dn
return 0
if #in = 1 and #dn = 1
return 1
if #in = 0 and #dn = 0 and #i = #d
return 1
return 0
END
To use this function, you can
declare #tmp table (a int, b int)
insert into #tmp values
(1,1),
(1,2),
(1,null),
(null,1),
(null,null)
---- in select ----
select *, [dbo].[ufn_equal_with_nulls](a,b) as [=] from #tmp
---- where equal ----
select *,'equal' as [Predicate] from #tmp where [dbo].[ufn_equal_with_nulls](a,b) = 1
---- where not equal ----
select *,'not equal' as [Predicate] from #tmp where [dbo].[ufn_equal_with_nulls](a,b) = 0
The results are:
---- in select ----
a b =
1 1 1
1 2 0
1 NULL 0
NULL 1 0
NULL NULL 1
---- where equal ----
1 1 equal
NULL NULL equal
---- where not equal ----
1 2 not equal
1 NULL not equal
NULL 1 not equal
The usage of sql_variant makes it compatible for variety of types
NULL is not anything...it is unknown. NULL does not equal anything. That is why you have to use the magic phrase IS NULL instead of = NULL in your SQL queries
You can refer this: http://weblogs.sqlteam.com/markc/archive/2009/06/08/60929.aspx
I am using the following query:
select count(*) from Table1 where CurrentDateTime>'2012-05-28 15:34:02.403504' and Error not in ('Timeout','Connection Error');
Surprisingly, this statement doesnot include the rows having Error value as NULL.My intention is to filter only rows with Error value as 'Timeout' (or) 'Connection Error'. I need to give an additional condition( OR Error is NULL) to retrieve the correct result.
Why is MYSQL filtering out results with NULL values?
I thought that IN keyword would return a boolean result (1/0) and now i understand that some MYSQL keywords doesnt return boolean values,it might return NULL too....but Why is it treating NULL as special?
This :
Error not in ('Timeout','Connection Error');
is semantically equivalent to:
Error <> 'TimeOut' AND Error <> 'Connection Error'
Rules about null comparison applies to IN too. So if the value of Error is NULL, the database can't make the expression true.
To fix, you could do this:
COALESCE(Error,'') not in ('Timeout','Connection Error');
Or better yet:
Error IS NULL OR Error not in ('Timeout','Connection Error');
Or more better yet:
CASE WHEN Error IS NULL THEN 1
ELSE Error not in ('Timeout','Connection Error') THEN 1
END = 1
OR doesn't short-circuit, CASE can somehow short-circuit your query
Perhaps a concrete example could illustrate why NULL NOT IN expression returns nothing:
Given this data: http://www.sqlfiddle.com/#!2/0d5da/11
create table tbl
(
msg varchar(100) null,
description varchar(100) not null
);
insert into tbl values
('hi', 'greet'),
(null, 'nothing');
And you do this expression:
select 'hulk' as x, msg, description
from tbl where msg not in ('bruce','banner');
That will output 'hi' only.
The NOT IN is translated as:
select 'hulk' as x, msg, description
from tbl where msg <> 'bruce' and msg <> 'banner';
NULL <> 'bruce' can't be determined, not even true, not even false
NULL <> 'banner' can't be determined, not even true not even false
So the null value expression, effectively resolved to:
can't be determined AND can't bedetermined
In fact, if your RDBMS supports boolean on SELECT(e.g. MySQL, Postgresql), you can see why: http://www.sqlfiddle.com/#!2/d41d8/828
select null <> 'Bruce'
That returns null.
This returns null too:
select null <> 'Bruce' and null <> 'Banner'
Given you are using NOT IN, which is basically an AND expression.
NULL AND NULL
Results to NULL. So it's like you are doing a: http://www.sqlfiddle.com/#!2/0d5da/12
select * from tbl where null
Nothing will be returned
Because null is undefined so null does not equal null. You always have to explicitly handle null.
IN returns NULL if the expression on the left hand side is NULL. In order to get the NULL values, you have to do:
select count(*) from Table1 where CurrentDateTime>'2012-05-28 15:34:02.403504' and (Error not in ('Timeout','Connection Error') or Error is null);
IN returns a trivalent BOOLEAN (which accepts NULL as a value). NOT IN returns the trivalent negation of IN, and negation of NULL is a NULL.
Imagine we have a table with all numbers from 1 to 1,000,000 in id and this query:
SELECT *
FROM mytable
WHERE id IN (1, 2, NULL)
or its equivalent:
SELECT *
FROM mytable
WHERE id = ANY
(
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT NULL
)
The predicate returns TRUE for 1 and 2 and NULL for all other values, so 1 and 2 are returned.
In its oppposite:
SELECT *
FROM mytable
WHERE id NOT IN (1, 2, NULL)
, or
SELECT *
FROM mytable
WHERE id <> ALL
(
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT NULL
)
, the predicate returns FALSE for 1 and 2 and NULL for all other values, so nothing is returned.
Note that boolean negation not only changes the operator (= to <>), but the quantifier too (ANY to ALL).
#Michael Buen ' s answer was the right answer for my case, but let me simplify why.
#Michael says in his post:
Error not in ('Timeout','Connection Error');
is semantically equivalent to:
Error <> 'TimeOut' AND Error <> 'Connection Error'
Rules about null comparison applies to IN too. So if the value of Error is NULL, the database can't make the expression true.
And in [1] I found this sentence which confirms his most important statement for understanding why IN fails with NULL. In the specifications ("specs") in [1] you will: "If one or both arguments are NULL, the result of the comparison is NULL, except for the NULL-safe <=> equality comparison operator."
So yeah, the thing is that sadly Mysql gets lost in such a case. I think Mysql designers shouldn't have done this, because when I compare 2 to NULL, Mysql SHOULD be able to see they are DIFFERENT, and not simply throwing mistaken results. For example, I did:
select id from TABLE where id not in (COLUMN WITH NULLS);
then it throws EMPTY results. BUT. If I do
select id from TABLE where id not in (COLUMN WITH OUT NULLS);
it shows the right result. So when using the IN operator, you must filter out the NULLS. This is not a desired behavior for me as a user, but it's documented in the specifications in [1]. I think that languages and technology should be simpler, in the sense that you should be able to DEDUCE without the need of reading the specs. And truly, 2 is DIFFERENT from NULL, I should be the one in charge of controlling and taking care of mistakes of a higher level of abstraction, but MySQL SHOULD throw a FALSE result when comparing NULL with a specific value.
References for the specs: [1] http://dev.mysql.com/doc/refman/5.6/en/type-conversion.html
Sorry for posting twice in the same forum, but I want to illustrate another example:
I agree with #Wagner Bianchi in [2] in this forum when he says:
<< It’s very trick when dealing with data and subqueries>>
Moreover, this should NOT be the behavior, I think Mysql's designers are mistaken when they made this decision documented in [1]. The design should be different. Let me explain: You know that when comparing
select (2) not in (1, 4, 3);
you will get:
+----------------------+
| (2) not in (1, 4, 3) |
+----------------------+
| 1 |
+----------------------+
1 row in set (0.00 sec)
BUT if in the list you have at least one NULL then:
select (2) not in (1, NULL, 3);
throws:
+-------------------------+
| (2) not in (1, NULL, 3) |
+-------------------------+
| NULL |
+-------------------------+
1 row in set (0.00 sec)
This is pretty absurd.
We are not the first ones in getting confused by this. See [2]
References:
[1] http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_in
[2] http://blog.9minutesnooze.com/sql-not-in-subquery-null/comment-page-1/#comment-86954
Why is this query returning 0 rows?
select t.f1, t.f2
from (select null f1, 'a' f2 from dual) t
where t.f1<>t.f2;
This is a distilled version of a complex query I have. I want to compare two tables containing one-to-one related data and I want to select those rows that contain different values for certain fields. But also there can be the case where one row is missing in one of the tables. The LEFT JOIN correctly returns null values for these rows, but then, the WHERE clause is incorrectly (or unexpectedly) filtering these rows out.
Why -in this case- 'null' IS NOT DIFFERENT to any not null value (like 'a') ?
What is driving me crazy is that this
select t.f1, t.f2
from (select null f1, 'a' f2 from dual) t;
returns 1 row (as I expected) but this
select t.f1, t.f2
from (select null f1, 'a' f2 from dual) t
where t.f1=t.f2;
returns 0 rows !! So null is not equal to 'a' and null is not different to 'a' !!
Please... Can anybody explain this?
Exactly. NULL represents an unknown value, not any specific value (it is not the same as NULL in C, or nil in Ruby, etc.) In SQL, if you compare something to the unknown value, the result is also unknown. And you will not get the rows where WHERE condition is unknown.
Try this:
SELECT NULL <> 2;
and you will see NULL as result.
Try this:
SELECT * FROM t WHERE NULL;
and no rows will come out, even if the table t is huge.
If you really need what you said you wanted (and I am not advocating this), you can do something like this:
SELECT T.f1, T.f2
FROM (SELECT NULL f1, 'a' f2) T
WHERE ((T.f1 IS NULL OR T.f2 IS NULL)
AND (T.f1 IS NOT NULL OR T.f2 IS NOT NULL))
OR T.f1 <> T.f2
The concept of NULL is a common source of confusion for newcomers to SQL, who often think that NULL is treated as the other values.
This is not the case. Conceptually, NULL means "a missing unknown value" and therefore it is treated very differently.
What you are seeing is pretty easy to explain. Consider the following example:
CREATE TABLE mytb (id int, value int);
INSERT INTO mytb VALUES (1, 100);
INSERT INTO mytb VALUES (2, 200);
INSERT INTO mytb VALUES (3, NULL);
INSERT INTO mytb VALUES (4, 400);
The above means that for the row with id = 3, the value is "unknown". It could be 300, or it could be 100, or anything else.
Therefore when you request the following:
SELECT * FROM mytb WHERE value <> 100;
+------+-------+
| id | value |
+------+-------+
| 2 | 200 |
| 4 | 400 |
+------+-------+
2 rows in set (0.00 sec)
The row with id = 3 is not returned, because NULL <> 100 returns "unknown". We don't know if row id = 3 has a value of 100, so the expression does not return true. I doesn't return false either. It returns "unknown" (NULL).
The condition of the WHERE clause can only be satisfied when the expression is true. When you compare something to NULL, the expression can never be true. It will be "unknown".
SQL NULL does not work the way you would like it to work: http://en.wikipedia.org/wiki/Sql_null
In short, NULL = NULL is not true. NULL <> NULL is not true. NULL <> 1 is not true. And so forth.
NULL value is nothing, it can't be equal or not equal to something.
If you want to check if your value is null - use "IS NULL" statement:
select t.f1, t.f2
from (select null f1, 'a' f2 from dual) t
where t.f1 IS NULL
If you want to check if your values are equal or not equal - you can use COALESCE function on nullable columns:
select t.f1, t.f2
from (select null f1, 'a' f2 from dual) t
where COALESCE(t.f1, '')<>COALESCE(t.f2, '');
Try to do this query:
select * from dual where NULL = NULL
It returns 0 rows. That's because to compare a value with null you have to do IS NULL or IS NOT NULL, otherwise it will return false.