I came across this strange behavior when I was hunting for a bug in a system. Consider following.
We have a mysql table which have varchar(100) column. See the following sql script.
create table user(`id` bigint(20) NOT NULL AUTO_INCREMENT,`user_id` varchar(100) NOT NULL,`username` varchar(255) DEFAULT NULL,PRIMARY KEY (`id`),UNIQUE KEY `user_id` (`user_id`)) ENGINE=InnoDB AUTO_INCREMENT=129 DEFAULT CHARSET=latin1;
insert into user(user_id, username) values('20120723145614834', 'user1');
insert into user(user_id, username) values('20120723151128642', 'user1');
When I execute following query I received 0 results.
select * from user where user_id=20120723145614834;
But When I execute following I get the result(note the single quote).
select * from user where user_id='20120723145614834';
This is expected since user_id field is varchar. Strange thing is that both following queries yield result.
select * from user where user_id=20120723151128642;
select * from user where user_id='20120723151128642';
Can anybody explain me the reason for this strange behavior. My MySql version is 5.1.63-0ubuntu0.11.10.1
Check mysql document 12.2. Type Conversion in Expression Evaluation
Comparisons that use floating-point numbers (or values that are
converted to floating-point numbers) are approximate because such
numbers are inexact. This might lead to results that appear
inconsistent:
mysql> SELECT '18015376320243458' = 18015376320243458;
-> 1
mysql> SELECT '18015376320243459' = 18015376320243459;
-> 0
So we better use always right data type for SQL.
Related
The Issue
I have a stored proc in a DB server that's bringing back a value of 5064803 when that record does not exist and the value should be 5064800 as per the query that builds the value of the variable.
I'm not sure if this is an issue with the value being of the FLOAT data type and the value in the record of the table ending in a double-zero or what but I cannot figure it out easily.
The table data types match those from the sensors that are set but this particular value from this sensor never actually gets set to a data type and it's usually always either a 1-8 digit INT with no decimal but I'd like to keep the data types the same as the correlated sensor just in case.
I've broke down the proc and I'm able to recreate the problem easily so I will post the detail below for those that may be able to help me figure out the issue and any workaround, etc.
The SQL Data
Create Table
delimiter $$
CREATE TABLE `number` (
`TimeInt` varchar(10) NOT NULL,
`TimeStr` datetime NOT NULL,
`IsInitValue` int(11) NOT NULL,
`Value` float NOT NULL,
`IQuality` int(11) NOT NULL,
UNIQUE KEY `uk_Times` (`TimeInt`,`TimeStr`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8$$
Insert Data
INSERT INTO `Number` (`TimeInt`,`TimeStr`,`IsInitValue`,`Value`,`IQuality`) VALUES ('1502618950','2017-08-13 10:09:10',1,5064800,0);
INSERT INTO `Number` (`TimeInt`,`TimeStr`,`IsInitValue`,`Value`,`IQuality`) VALUES ('1502618796','2017-08-13 10:06:36',0,5064800,3);
INSERT INTO `Number` (`TimeInt`,`TimeStr`,`IsInitValue`,`Value`,`IQuality`) VALUES ('1502617167','2017-08-13 09:39:27',1,5063310,0);
INSERT INTO `Number` (`TimeInt`,`TimeStr`,`IsInitValue`,`Value`,`IQuality`) VALUES ('1502613355','2017-08-13 08:35:55',0,5063310,3);
INSERT INTO `Number` (`TimeInt`,`TimeStr`,`IsInitValue`,`Value`,`IQuality`) VALUES ('1502612814','2017-08-13 08:26:54',1,0,0);
INSERT INTO `Number` (`TimeInt`,`TimeStr`,`IsInitValue`,`Value`,`IQuality`) VALUES ('1502609015','2017-08-13 07:23:35',0,0,3);
The SQL Query Breakdown
SET #bStartTime = '2017-08-13 09:24:16';
SET #bEndTime = '2017-08-13 10:06:31';
SET #LastNumber = (SELECT Value FROM Number ORDER BY TimeStr DESC LIMIT 1);
SET #NowNumber = (SELECT Value FROM Number WHERE TimeStr BETWEEN #bStartTime AND #bEndTime ORDER BY TimeStr DESC LIMIT 1);
SELECT #NowNumber;
SELECT #LastNumber;
Recreating the Issue
So based on The SQL Query Breakdown above, once all the data is in the table and then I run the queries within the SELECT queries alone within the #NowNumber and/or #LastNumber variables, I get the correct result of 5064800. However, if I run the entire SET statements for both of those to have it set the query and then just do a SELECT of those variable, it brings back the wrong result of 5064803.
So for example if I run SELECT Value FROM Number ORDER BY TimeStr DESC LIMIT 1 then the correct value is returned. If I run SET #LastNumber = (SELECT Value FROM Number ORDER BY TimeStr DESC LIMIT 1); and then run SELECT #LastNumber; I get the incorrect value returned.
Server System Specs
This particular MySQL Server is running the x86 version of 5.5.50 on Windows Server 2008 with 144 GB of RAM for some quick specs.
Question
I'd like to know what is causing this, and if there is a workaround to the problem either with or without changing the data type of the column assuming that's the issue when it's returned as a variable rather than just a straight query result.
I'll be happy to disclose more technical specs of the environment if needed but I've included what I think it important for the question. Perhaps this is a version bug or there's something obvious that causes this that I cannot see easily so I'm hoping someone can help me with this or explain why this is or is not possible with MySQL.
Sorry, declares can only be used in stored procedures in MySQL. I found this article which may help. It explains how MySQL rounds when storing digits and recommends using doubles. Try changing your floats to doubles.
MySql FLOAT datatype and problems with more then 7 digit scale
I'm having a problem with MySQL returning the incorrect result when applying a WHERE condition to an integer field with a string value.
CREATE TABLE `people` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1;
INSERT INTO `people` (`id`, `name`)
VALUES
(1, 'Bob'),
(2, 'Sally'),
(3, 'Jim');
Now when I run the query:
SELECT *
FROM people
WHERE id = '1-abcd';
My result set is:
id name
1 Bob
MySQL appears to be truncating the string value '1-abcd' to '1' behind the scenes as soon as it hits a non-integral character (in the conversion from a string to INT).
You're probably wondering why this matters. I'm trying to fix a site for a PCI compliance scan. The scan thinks the URI '/some/page?id=102-1' is allowing some form of sequel injection, but in reality it's showing the same content at '/some/page?id=102'.
This is not an issue in one place. It is an issue all over the place, and it's a fairly large system. Is there some way to rectify this on the MySQL end of things, so it no longer mistakenly judges the two values to be equivalent? I looked at the documentation for SQL modes, but didn't see anything regarding this circumstance.
UPDATE: I filed a dispute with the company that produced the scan, which they accepted, so I'm no longer in the woods. But it is disappointing that there's apparently no way to configure the casting behavior of MySQL from a string to INT in this case. (You can, but only for INSERTs and UPDATEs.)
What happens that MySQL type-casts the string literal value to an integer, and when it does that it starts from the left of the string and as soon as it reaches a character that cannot be considered part of a number, it strips out everything from that point on. So 1-0 gives output matching to 1. To do this you can use cast. I am not 100% sure about the syntax but it is like this:
select * from people
where id =
(
case when ISNUMERIC( '1-0' )
then cast ('1-0' as int)
else null
end )
What this will do is that if it is an numeric value then it will return the correct matching row or else not.
Edit:
The above query seems to be of MSSQL/Oracle and would not work with MySQL. For MySQL you can use RegExp. I have never use one but you can find more details here:
http://mysqlhints.blogspot.in/2012/01/how-to-find-out-if-entire-string-is.html
http://www.ash.burton.fm/blogs/2010/12/quick-tip-mysql-equivalent-of-isnumeric
http://www.justskins.com/forums/how-to-use-isnumeric-137604.html
In another post on stackoverflow, I read that INSTR could be used to order results by relevance.
My understanding of col LIKE '%str%' andINSTR(col, 'str')` is that they both behave the same. There seems to be a difference in how collations are handled.
CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(64) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
INSERT INTO users (name)
VALUES ('Joël'), ('René');
SELECT * FROM users WHERE name LIKE '%joel%'; -- 1 record returned
SELECT * FROM users WHERE name LIKE '%rene%'; -- 1 record returned
SELECT * FROM users WHERE INSTR(name, 'joel') > 0; -- 0 records returned
SELECT * FROM users WHERE INSTR(name, 'rene') > 0; -- 0 records returned
SELECT * FROM users WHERE INSTR(name, 'joël') > 0; -- 1 record returned
SELECT * FROM users WHERE INSTR(name, 'rené') > 0; -- 1 record returned
Although INSTR does some conversion, it finds ë in é.
SELECT INSTR('é', 'ë'), INSTR('é', 'e'), INSTR('e', 'ë');
-- returns 1, 0, 0
Am I missing something?
http://sqlfiddle.com/#!2/9bf21/6 (using mysql-version: 5.5.22)
This is due to bug 70767 on LOCATE() and INSTR(), which has been verified.
Though the INSTR() documentation states that it can be used for multi-byte strings, it doesn't seem to work, as you note, with collations like utf8_general_ci, which should be case and accent insensitive
This function is multi-byte safe, and is case sensitive only if at least one argument is a binary string.
The bug report states that although MySQL does this correctly it only does so when the number of bytes is also identical:
However, you can easily observe that they do not (completely) respect collations when looking for one string inside another one. It seems that what's happening is that MySQL looks for a substring which is collation-equal to the target which has exactly the same length in bytes as the target. This is only rarely true.
To pervert the reports example, if you create the following table:
create table t ( needle varchar(10), haystack varchar(10)
) COLLATE=utf8_general_ci;
insert into t values ("A", "a"), ("A", "XaX");
insert into t values ("A", "á"), ("A", "XáX");
insert into t values ("Á", "a"), ("Á", "XaX");
insert into t values ("Å", "á"), ("Å", "XáX");
then run this query, you can see the same behaviour demonstrated:
select needle
, haystack
, needle=haystack as `=`
, haystack LIKE CONCAT('%',needle,'%') as `like`
, instr(needle, haystack) as `instr`
from t;
SQL Fiddle
Currently we have an interessting problem regarding the sort order of MySQL in an enum-field. The fields enum entries have been sorted in the order we want it. Just to be save, we added a CONCAT around it, so it would be cast to char and ordered in alphabetical order, just as suggested by the MySQL-reference (MySQL Reference - Enum)
Make sure that the column is sorted lexically rather than by index number by coding ORDER BY CAST(col AS CHAR) or ORDER BY CONCAT(col).
But that didn't produce the expected results, so we started to investigate further. It seems that the order by statement doesn't work on a combination of enum and the concat function. I've wrote the following sample script, which should show my point:
CREATE TABLE test (
`col1` enum('a','b','c') COLLATE utf8_bin DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
INSERT INTO test
VALUES ('b'), ('c'), ('a');
SELECT * FROM test; -- b, c, a
SELECT * FROM test ORDER BY col1 ASC; -- a, b, c
SELECT * FROM test ORDER BY CAST(col1 AS CHAR) ASC; -- a, b, c
SELECT * FROM test ORDER BY CAST(col1 AS BINARY) ASC; -- a, b, c
SELECT * FROM test ORDER BY CONCAT(col1) ASC; -- b, c, a - This goes wrong
I am currently suspecting some kind of problem with the collation/encoding, but I'm not sure. My databases default encoding is also utf8. The MySQL version is 5.6.12 but it seems to be reproduceable with MySQL 5.1. The storage engine is MyIsam but it also occurs with the memory engine.
Any help would be appreciated.
Update:
As it seems the problem is produced only in MySQL 5.6 and by the collation of the column. With the first CREATE TABLE statement, the queries work fine.
CREATE TABLE test (
`col1` enum('a','b','c') COLLATE utf8_general_ci DEFAULT NULL
)
With the second they don't.
CREATE TABLE test (
`col1` enum('a','b','c') COLLATE utf8_bin DEFAULT NULL
)
The collation of the table and/or database don't seem to affect the queries. The queries can be tested in this SQL Fiddle
Strange,it works in this fiddle.Do you have a trigger or something?
http://sqlfiddle.com/#!2/0976a/2
BUT,in 5.6 goes haywire:
http://sqlfiddle.com/#!9/0976a/1
Mysql bug,probably.
More,if you input the values in the enum in the "proper" order it works:
http://sqlfiddle.com/#!9/a3784/1
IN the doc:
ENUM values are sorted based on their index numbers, which depend on
the order in which the enumeration members were listed in the column
specification. For example, 'b' sorts before 'a' for ENUM('b', 'a').
As per the document:
Under the Handling of Enumeration Literals section, it states that:
If you store a number into an ENUM column, the number is treated as
the index into the possible values, and the value stored is the
enumeration member with that index. (However, this does not work with
LOAD DATA, which treats all input as strings.) If the numeric value is
quoted, it is still interpreted as an index if there is no matching
string in the list of enumeration values. For these reasons, it is not
advisable to define an ENUM column with enumeration values that look
like numbers, because this can easily become confusing.
For example, the following column has enumeration members with string values of '0', '1', and '2', but numeric index values of 1, 2, and 3:
numbers ENUM('0','1','2')
If you store 2, it is interpreted as an
index value, and becomes '1' (the value with index 2). If you store
'2', it matches an enumeration value, so it is stored as '2'. If you
store '3', it does not match any enumeration value, so it is treated
as an index and becomes '2' (the value with index 3).
mysql> INSERT INTO t (numbers) VALUES(2),('2'),('3');
mysql> SELECT * FROM t;
+---------+
| numbers |
+---------+
| 1 |
| 2 |
| 2 |
+---------+
In your case:
INSERT INTO test
VALUES ('2'), ('3'), ('1');
Index value of '2' is 2, '3' is 3 and '1' is 1.
So the output is 2,3,1
I am writing a flexible search mechanism for a customer's website. I am utilizing union clauses to query a number of different fields in the database in search of a string value entered by the user. This works fine except for one issue.
When comparing a string of a text to an integer that is currently set to zero, the match always returns true. In other words, according to MySQL, "email#example.com" is equal to 0.
I have tried utilizing the CAST and CONVERT function to turn this into a standard string to string comparison, but I can't seem to get the syntax right. My attempts either repeat the above issue or return no rows at all when some should match. I am also concerned that doing this would have an effect on performance since I am combining lots of unions.
What I really need is a strict comparison between an entered string and the value in the database, be it an integer or string.
EDIT:
Here is an example.
CREATE TABLE `test_table` (
`id` INT NOT NULL AUTO_INCREMENT ,
`email` VARCHAR(255) NOT NULL ,
`phone` BIGINT(19) NOT NULL DEFAULT '0' ,
PRIMARY KEY (`id`) )
ENGINE = MyISAM;
INSERT INTO `test_table` (`id`, `email`, `phone`) VALUES (1, 'email#example.com', 0);
SELECT * FROM test_table WHERE phone = 'email#example.com';
Execute this and the one row that has been inserted will return. My issue is that it shouldn't!
This query should fail:
SELECT * FROM test_table WHERE cast(phone as char) = 'email#example.com';
The cause of the original problem is that when comparing strings and numbers, it converts the string to a number (so you can write where phone = '123'). You need to use an explicit cast of the field to make it a string-to-string comparison, to prevent this default conversion.
Unfortunately, casting like this is likely to prevent it from using indexes. Even if the field is already char, the cast apparently prevents it from indexing.
You could also solve it during input validation: if phone is an integer, don't allow the user to provide a non-integer value in the search field.
How about replacing:
SELECT * FROM test_table WHERE phone = 'email#example.com'
with:
SELECT * FROM test_table WHERE phone = 'email#example.com' and phone <> 0
<> means different from.
This will work for you because you are using 0 in the phone column to mean there isn't a phone number (although it would be better style to use NULL for no phone number).