Could somebody tell me how I can detect if a cast failed in MySQL using CAST() function?
These two lines return the same value: 0.
SELECT CAST('Banana' AS UNSIGNED INTEGER) AS 'CAST1';
SELECT CAST('0' AS UNSIGNED INTEGER) AS 'CAST2';
You can use regular expressions to validate the data before the conversion:
select (case when val regexp '^[0-9]+$' then cast(val as unsigned integer) end)
The SHOW WARNINGS statement and the ##WARNINGS system variable are the built in methods to do this. There is no mechanism to automatically upgrade all warnings to errors, but there are some things you can do.
You may want to start MySQL with the --show-warnings option, although that might just display the count of warnings with the row count. I can't recall anymore. I don't know if there is a my.ini option for this option. There's also the --log-warnings option, which I believe does have an option in the ini/cnf file. If you're executing a script or using the CLI, the \W command turns show warnings on and \w turns them off for (IIRC) the current connection.
You may also want to look at the SQL mode. TRADITIONAL is probably the most like a normal RDBMS, but it's kind of a rats nest of options. The STRICT modes are what you're most likely after, but read through that page. Most apps built on MySQL take advantage of the (non-deterministic) GROUP BY extensions that bite just about everybody moving to or away from MySQL, and TRADITIONAL enables ONLY_FULL_GROUP_BY, which effectively disables those extensions and the RDBMS doesn't support the OVER() clause. I don't know if silently succeeding at typecasting will abort a transaction even in traditional/strict mode, however.
MySQL is kind of a mine field of these kinds of issues (e.g., zero dates) so it kind of has a poor reputation with DBAs, especially those who worked with v3.x or v4.x.
You could e.g. check the warning_count variable:
MySQL [test]> SELECT CAST(0 AS UNSIGNED INTEGER) AS 'CAST1', ##warning_count;
+-------+-----------------+
| CAST1 | ##warning_count |
+-------+-----------------+
| 0 | 0 |
+-------+-----------------+
1 row in set (0.01 sec)
MySQL [test]> SELECT CAST('Banana' AS UNSIGNED INTEGER) AS 'CAST1', ##warning_count;
+-------+-----------------+
| CAST1 | ##warning_count |
+-------+-----------------+
| 0 | 1 |
+-------+-----------------+
1 row in set, 1 warning (0.00 sec)
There's a caveate though: the warning count is only reset per statement, not per result row,
so if CAST() gets executed mutiple times, e.g. for each result row, the counter will go up
on each failed invocation.
Also warnings don't seem to get reset on successful queries that don't touch any tables,
so in the example above a 2nd
SELECT CAST(0 AS UNSIGNED INTEGER) AS 'CAST1', ##warning_count;
will still show 1 warning, while e.g.
SELECT CAST(0 AS UNSIGNED INTEGER) AS 'CAST1', ##warning_count
FROM mysql.user LIMIT 1;
will correctly reset it to 0 ...
Well, you could incorporate ##warning_count but somehow create workaround for its buggy functionality.
Take a look at this code below. Yes, I know it's ugly, but it works.
SELECT
IF(WarningCount = 0, ConversionResult, NULL)
FROM (
SELECT
CAST('banana' AS DECIMAL(10, 6)) AS ConversionResult
, ##warning_count AS WarningCount
FROM <any non empty table>
LIMIT 1
) AS i;
In inner SELECT I'm getting 1 row (LIMIT 1) from any existing table. I'm converting string ('banana') and get WarningCount. In outer SELECT I'm checking WorkingCount and if it's equal to 0 (conversion successful) then returning converted value.
I would suggest to use such function:
drop function if exists to_number;
delimiter $$
create function to_number (number varchar(10)) returns int
begin
declare error_message varchar(45);
if (number regexp ('^[+-]?[0-9]*([0-9]\\.|[0-9]|\\.[0-9])[0-9]*(e[+-]?[0-9]+)?$'))
then
return number;
else
set error_message = concat('The given value "', number, '" is not a number.');
signal sqlstate '45000' set message_text = error_message;
end if;
end;
It rises an error if the given value is not number or returns the number value.
If you are trying to determine how many values in a varchar are numbers you can try:
select count(*),
sum(is_num)
from (select case when cast(cast(ar_number as unsigned) as char) = ar_number then 1 else 0 end as is_num
from the_table) as t1;
SQL Server supports the try_cast function
Related
I am wondering if it is possible to avoid a second equal calculation within a CASE statement of MySQL 5.7?
CASE
WHEN char_length(cat.DESCRIPTION) > 0 THEN char_length(cat.DESCRIPTION)
ELSE ''
END AS D_LENGTH
The second char_length seems redundant to me and might be reducing query performance. Is there a way to improve this?
Since you seem to want to display empty string when the character length of the column be zero, you could try using a TRIM trick here:
SELECT TRIM(LEADING '0' FROM CHAR_LENGTH(cat.DESCRIPTION)) AS D_LENGTH
FROM yourTable cat;
This works because whenever the character length of the description be greater than zero, it would never have any leading zeroes. When that length is actually zero, the call to TRIM above would just strip off the single zero, leaving behind an empty string.
Regarding your current approach, no, there isn't much you can do directly to avoid the double call to CHAR_LENGTH. But, as shown above, there are ways out which completely avoid the duplication.
You may try to use intermediate user-defined variable:
CASE
WHEN (#tmp:=char_length(cat.DESCRIPTION)) > 0
THEN #tmp
ELSE ''
END AS D_LENGTH
On "clear" model (a table with one varchar column, data lengths 20-250, 1kk rows, no indices, the whole table is cached) it takes ~15% less time to execute on my system (5.11-5.32s against 5.97-6.23s).
The query produces a warning "1287 Setting user variables within expressions is deprecated and will be removed in a future release. Consider alternatives: 'SET variable=expression, ...', or 'SELECT expression(s) INTO variables(s)'." - ignore it.
First, the performance cost of evaluating any expression is much less than the cost of the rest of the query. So don't bother with such optimization.
Second, to rise to the challenge:
mysql> SELECT COALESCE(NULLIF(CHAR_LENGTH('asdf'), 0), 'empty');
+---------------------------------------------------+
| COALESCE(NULLIF(CHAR_LENGTH('asdf'), 0), 'empty') |
+---------------------------------------------------+
| 4 |
+---------------------------------------------------+
1 row in set (0.00 sec)
mysql> SELECT COALESCE(NULLIF(CHAR_LENGTH(''), 0), 'empty');
+-----------------------------------------------+
| COALESCE(NULLIF(CHAR_LENGTH(''), 0), 'empty') |
+-----------------------------------------------+
| empty |
+-----------------------------------------------+
1 row in set (0.00 sec)
Here is the code
mysql> SELECT id FROM tbl WHERE id = '1h';
+----+
| id |
+----+
| 1 |
+----+
1 row in set
There is indeed a field with id 1 (but not '1h').
Here is an extraction from MySQL docs: http://dev.mysql.com/doc/refman/5.1/en/type-conversion.html
mysql> SELECT 1 > '6x';
-> 0
mysql> SELECT 7 > '6x';
-> 1
So this bug is documented, so to say. The question is what's the reason for such behavior and how to correct it to make this not cast strings with char symbols? I can cast all field values like
mysql> SELECT id FROM tbl WHERE cast(`id`, BINARY) = '1h';
but i don't like this variant too much
This is not a bug.
The solution is not to query on numeric columns using a string value for your condition.
Never rely on implicit type casting.
None of your observations are bugs. They are the result of relying in implicit type casting.
In all of your examples, you're requiring MySQL to convert a string to an int. If you read the very page that you linked to, you will see that MySQL follows some rules in achieving this. As a result
'1h' -> 1
'6x' -> 6
'x6' -> 0
So, if you follow these rules, you'll be OK.
Better still, just don't put MySQL in a position where it needs to be doing these conversion. Such situations usually point to some kind of logic bug elsewhere in the system.
I am developing a system using MySQL queries written by another programmer, and am adapting his code.
I have three questions:
1.
One of the queries has this select statement:
SELECT
[...]
AVG(mytable.foo, 1) AS 'myaverage'`,
Is the 1 in AVG(mytable.foo, 1) AS 'myaverage' legitimate? I can find no documentation to support its usage?
2.
The result of this gives me average values to 2 decimal places, why?.
3.
I am using this to create a temp table. So:
(SELECT
[...]
AVG(`mytable`.`foo`, 1) AS `myaverage`,
FROM
[...]
WHERE
[...]
GROUP BY
[...])
UNION
(SELECT
[...]
FROM
[...]
WHERE
[...]
GROUP BY
[...])
) AS `tmptable`
ORDER BY
`tmptable`.`myaverage` DESC
When I sort the table on this column I get output which indicates that this average is being stored as a string, so the result is like:
9.3
11.1
In order to get around this what should I use?
Should I be using CAST or CONVERT, as DECIMAL (which I read is basically binary), BINARY itself, or UNSIGNED?
Or, is there a way to state that myaverage should be an integer when I name it in the AS statement?
Something like:
SELECT
AVG(myaverage) AS `myaverage`, INT(10)
Thanks.
On your last question: can you post the exact MySQL query that you are using?
The result type of a column from a UNION is determined by everything you get back. See http://dev.mysql.com/doc/refman/5.0/en/union.html .
So, even if your AVG() function returns a DOUBLE, the other part of the UNION may still return a string. In which case the column type of the result will be a string.
See the following example:
mysql> select a from (select 19 as a union select '120') c order by a;
+-----+
| a |
+-----+
| 120 |
| 19 |
+-----+
2 rows in set (0.00 sec)
mysql> select a from (select 19 as a union select 120) c order by a;
+-----+
| a |
+-----+
| 19 |
| 120 |
+-----+
2 rows in set (0.00 sec)
Just for anyone who's interested, I must have deleted or changed my predecessors code so this AVG question was incorrect. The correct code was ROUND(AVG(myaverage),1). Apologies to those who scrathed their heads over my stupidity.
on 1.
AVG() accepts exactly one argument, otherwise MySQL will raise an error:
mysql> SELECT AVG( id, 1 ) FROM anytable;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' 1 )' at line 1
http://dev.mysql.com/doc/refman/5.1/en/group-by-functions.html#function_avg
Just because I'm curious - what should the second argument do?
I recently just fixed a bug in some of my code and was hoping someone could explain to me why the bug occurred.
I had a query like this:
SELECT * FROM my_table WHERE my_field=13
Unexpectedly, this was returning rows where my_field was equal to either 13 or 13a. The fix was simple, I changed the query to:
SELECT * FROM my_table WHERE my_field='13'
My question is, is this supposed to be the case? I've always thought that to return a similar field, you would use something like:
SELECT * FROM my_table WHERE my_field LIKE '13%'
What is the difference between LIKE + a Wild Card vs an equals operator with no quotes?
This statement returns rows for my_field = '13a':
SELECT * FROM my_table WHERE my_field=13
Because MySQL performs type conversion from string to number during the comparison, turning '13a' to 13. More on that in this documentation page.
Adding quotes turns the integer to a string, so MySQL only performs string comparison. Obviously, '13' cannot be equal to '13a'.
The LIKE clause always performs string comparison (unless either one of the operands is NULL, in which case the result is NULL).
My guess would be that since you didn't enclose it in quotes, and the column was a char/varchar column, MySQL tried to do an implicit conversion of the varchar column to an int.
If one of the rows in that table contained a value that couldn't be converted to an int, you would probably get an error. Also, because of the conversion, any indexes you might have had on that column would not be used either.
This has to do with types and type conversion. With my_field=13 , 13 is an integer, while my_field is in your case likely some form of text/string. In such a case, mysql will try to convert both to a floating point number and compare those.
So mysql tries to convert e,g, "13a" to a float, which will which be 13, and 13 = 13
In my_field = '13' , both operands are text and will be compared as text using =
In my_field like '13%' both operands are also text and will be compared as such using LIKE, where the special % means a wildcard.
You can read about the type conversion mysql uses here.
This is because the MySQL type conversion works this way. See here: http://dev.mysql.com/doc/refman/5.0/en/type-conversion.html
It releases a warning as well. see the code below
mysql> select 12 = '12bibo';
+---------------+
| 12 = '12bibo' |
+---------------+
| 1 |
+---------------+
1 row in set, 1 warning (0.00 sec)
mysql> show warnings;
+---------+------+--------------------------------------------+
| Level | Code | Message |
+---------+------+--------------------------------------------+
| Warning | 1292 | Truncated incorrect DOUBLE value: '12bibo' |
+---------+------+--------------------------------------------+
1 row in set (0.00 sec)
Looks like someone raised a bug as well: http://bugs.mysql.com/bug.php?id=42241
I have a string that is defined as one or more dot-separated integers like 12345, 543.21, 109.87.654, etc. I'm storing values in a MySQL database and then need to find the rows that compare with a provided value. What I want is to select rows by comparing each component of the string against the corresponding component of the input string. With standard string comparison in MySQL, here's where this breaks down:
mysql> SELECT '543.21' >= '500.21'
-> 1
mysql> SELECT '543.21' >= '5000.21'
-> 1
This is natural because the string comparison is a "dictionary" comparison that doesn't account for string length, but I want a 0 result on the second query.
Is there a way to provide some hint to MySQL on how to compare these? Otherwise, is there a way to hint to ActiveRecord how to do this for me? Right now, the best solution I have come up with is to select all the rows and then filter the results using Ruby's split and reject methods. (The entire data set is quite small and not likely to grow terribly much for the foreseeable future, so it is a reasonable option, but if there's a simpler way I'm not considering I'd be glad to know it.)
You can use REPLACE to remove dots and CAST to convert string to integer:
SELECT CAST(REPLACE("543.21", ".", "") AS SIGNED) >= CAST(REPLACE("5000.21", ".", "") AS SIGNED)
mysql> SELECT '543.21' >= '5000.21';
+-----------------------+
| '543.21' >= '5000.21' |
+-----------------------+
| 1 |
+-----------------------+
1 row in set (0.00 sec)
mysql> SELECT '543.21'+0 >= '5000.21'+0;
+---------------------------+
| '543.21'+0 >= '5000.21'+0 |
+---------------------------+
| 0 |
+---------------------------+
1 row in set (0.00 sec)
This indeed only works for valid floats. Doing it for more then 1 dot would require a LOT of comparing of SUBSTRING_INDEX(SUBSTRING_INDEX(field, '.', <positionnumber you're comparing>), '.', -1) (with a manual repeat for the maximum number of position's you are comparing)