It's frustrated with MySQL's pattern escaping used in LIKE operator.
root#dev> create table foo(name varchar(255));
Query OK, 0 rows affected (0.02 sec)
root#dev> insert into foo values('with\\slash');
Query OK, 1 row affected (0.00 sec)
root#dev> insert into foo values('\\slash');
Query OK, 1 row affected (0.00 sec)
root#dev> select * from foo where name like '%\\\\%';
Empty set (0.01 sec)
root#dev> select * from foo;
+------------+
| name |
+------------+
| with\slash |
| \slash |
+------------+
2 rows in set (0.00 sec)
root#dev> select * from foo where name like '%\\\\%';
Empty set (0.00 sec)
root#dev> select * from foo where name like binary '%\\\\%';
+------------+
| name |
+------------+
| with\slash |
| \slash |
+------------+
2 rows in set (0.00 sec)
According to MySQL docs: http://dev.mysql.com/doc/refman/5.5/en/string-comparison-functions.html#operator_like
%\\\\% is the right operand, but why it yields no result?
EDIT:
The database I'm testing that in has character_set_database set to utf8. To further my investigation, I created the same setup in a database that has character_set_database set to latin1, and guess what, '%\\\\%' works!
EDIT:
The problem can be reproduced and it's the field collation problem. Details: http://bugs.mysql.com/bug.php?id=63829
In MySQL 5.6.10, with the text field collation utf8mb4_unicode_520_ci this can be achieved by using 5 backslash characters instead of 4, i.e:
select * from foo where name like binary '%\\\\\%';
Somehow, against all expectations, this properly finds all rows with backslashes.
At least this should work until the MySQL field collation bug above is fixed. Considering it's been more than 5 years since the bug is discovered, any app designed with this may outlive its usefulness before MySQL is even fixed - so should be a pretty reliable workaround.
With MySQL 5.0.12 dev on Windows 10 I got the following results when I changed the query from
SELECT * FROM `foo` WHERE `name` LIKE '%http:\/\/%'
to
SELECT * FROM `foo` WHERE `name` LIKE '%http:\\\\\\\%'
it works and yet the first string with forward slashes was the original field content. It seems to have interpreted forward slashes as backslashes.
It seems it has some relation to that MySQL bug: http://bugs.mysql.com/bug.php?id=46659
I think you connect to mysql not specifying correct --character-set-server option (which defaults to latin1 with collation latin1_swedish_ci), and having utf-8 as the current charset of the console. That causes incorrect char conversions and comparisons when you deal with data which supposed to be converted to the utf8 from the charset of --character-set-server.
Related
I'm trying to update a mariadb table column, to a string that contains a literal backslash.
I want the resulting string in the table to be
4.4 \(blah blah\)
I've tried
UPDATE table SET string = '4.4 \\(blah blah\\)' WHERE string = '4.4 (blah blah)';
This works when I run it in Sequel Pro, but when I run it as part of a ruby/rails migration, the result is that the column remains unchanged, ie. 4.4 (blah blah).
I've tried every combination of single quotes, double quotes, single backslash, double backslash. I also tried a triple backslash.
NO_BACKSLASH_ESCAPES sql_mode.
Enabling this mode disables the use of the backslash character () as
an escape character within strings and identifiers. With this mode
enabled, backslash becomes an ordinary character like any other, and
the default escape sequence for LIKE expressions is changed so that no
escape character is used.
mysql> create table my_table (
-> string varchar(255) );
Query OK, 0 rows affected (0.34 sec)
mysql>
mysql> insert into my_table values
-> ('4.4 (blah blah)');
Query OK, 1 row affected (0.07 sec)
mysql> select ##sql_mode;
+------------------------+
| ##sql_mode |
+------------------------+
| NO_ENGINE_SUBSTITUTION |
+------------------------+
1 row in set (1.318 sec)
mysql> set session sql_mode='NO_BACKSLASH_ESCAPES,NO_ENGINE_SUBSTITUTION';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> UPDATE my_table SET string = '4.4 \(blah blah\)' WHERE string = '4.4 (blah blah)';
Query OK, 1 row affected (0.08 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select * from my_table;
+-------------------+
| string |
+-------------------+
| 4.4 \(blah blah\) |
+-------------------+
1 row in set (0.02 sec)
I'm trying to use MySQL 8.0 but I'm having some problems. I have installed MySQL 5.7 and 8.0, and have different behavior with CHAR columns.
For MySQL 5.7:
mysql> create table test (id integer, c5 char(5));
Query OK, 0 rows affected (0.00 sec)
mysql> insert into test values(0, 'a');
Query OK, 1 row affected (0.00 sec)
mysql> select * from test where c5 = 'a ';
+------+------+
| id | c5 |
+------+------+
| 0 | a |
+------+------+
1 row in set (0.00 sec)
mysql>
For MySQL 8.0:
mysql> create table test (id integer, c5 char(5));
Query OK, 0 rows affected (0.01 sec)
mysql> insert into test values(0, 'a');
Query OK, 1 row affected (0.01 sec)
mysql> select * from test where c5 = 'a ';
Empty set (0.00 sec)
mysql>
Both servers have same configuration.
MySQL 5.7:
[mysqld]
port=3357
datadir=/opt/mysql_57/data
sql_mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION"
default_storage_engine=innodb
character-set_server=utf8mb4
socket=/opt/mysql_57/mysql57.sock
max_allowed_packet=4194304
server_id=1
lower_case_table_names=0
MySQL 8.0:
[mysqld]
port=3380
datadir=/opt/mysql_80/data
sql_mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION"
default_storage_engine=innodb
character-set_server=utf8mb4
socket=/opt/mysql_80/mysql80.sock
max_allowed_packet=4194304
server_id=1
lower_case_table_names=0
A brief overview of the MySQL 8.0 changelog didn't give me any information. Where described this behavior changes?
Best regards.
How MySQL handled trailing spaces, depends on the collation being used. See https://dev.mysql.com/doc/refman/8.0/en/charset-binary-collations.html for details.
What has changed between 5.7 and 8.0, is that the default character set is now UTF8mb4 with NOPAD collations.
If you want another behavior, you should change character set/collation for your column/table/database. Check the INFORMATION_SCHEMA table COLLATIONS for available PAD collations. (One warning: The older PAD SPACE collations may be less efficient. Quite some work has been made to improve the performance of the new Unicode collations based on UCA 9.0.0.)
See PAD_CHAR_TO_FULL_LENGTH in MySQL documentation
I've got a table which contains a varchar(1024) field, which in that contains strings which has hex encoded strings. This table is filled automatically and I have to provide an SP to allow users to download this, therefore, I need to change the hex back into human readable form.
If I manually run this statement (taking the Hex data from the field), it works just fine:
SELECT X'5468697320697320612074657374206D6573736167652031323334353637383930';
But I cannot find a working example of getting this to work when calling the field/column name. I've found a few examples, but these just return a null or 0.
I've tried X and UnHex() and neither give me a result.
Where am I going wrong?
Thanks
EDIT:
Okay, after doing a bit more testing, it appears it must be the way it's being written to the database in the first place.
It's a Classic ASP page that calls an SP, which creates the database entry. In this method, the write to the DB works, and I can see the HEX content in the field. Copying the content of the field, and putting this into a Select X'123123' gives me the ASCII values, as I want.
If I try this as a Select, this fails, giving me a zero or Null return.
SELECT Message_Body_Hex, UNHEX(Message_Body_Hex) FROM messages_inbound
returns:
Message_Body_Hex......unhex(Message_Body_Hex)
417265612032........(NULL)
Still confused! :)
I realize this is an old question but I ran into this same problem today and solved it using a combination of HEX and CAST. Using your example, try this:
SELECT HEX(CAST(X'5468697320697320612074657374206D6573736167652031323334353637383930' AS CHAR(33)))
When pulling from a table you'd substitute the field name:
SELECT HEX(CAST(binary_field AS CHAR(33)))
I've seen other answers recommending to use MAX in place of the 33 but this appears to work fine. Here are some sources I used:
SQL Server converting varbinary to string
and
How to convert from varbinary to char/varchar in mysql
Using the UNHEX() function seems to work fine on MySQL 5.5.29-1:
mysql> create table t1 ( f1 varchar(1024) );
Query OK, 0 rows affected (0.03 sec)
mysql> insert into t1 values('5468697320697320612074657374206D6573736167652031323334353637383930');
Query OK, 1 row affected (0.02 sec)
mysql> select f1 from t1;
+--------------------------------------------------------------------+
| f1 |
+--------------------------------------------------------------------+
| 5468697320697320612074657374206D6573736167652031323334353637383930 |
+--------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> select unhex(f1) from t1;
+-----------------------------------+
| unhex(f1) |
+-----------------------------------+
| This is a test message 1234567890 |
+-----------------------------------+
1 row in set (0.00 sec)
If I try to search for a value in mysql database and the string value contains dot in it, query returns 0 rows. Example:
SELECT * FROM table WHERE `username`='marco.polo' --> 0 rows
SELECT * FROM table WHERE `username` LIKE '%.polo%' --> 0 rows
SELECT * FROM table WHERE `username` LIKE 'polo' --> Success
This appeared after moving server and database to another place. I know that dot is a set of extended regular expressions, but it should not apply to equal nor LIKE operator, simply because I don't use REGEXP in query.
I've tested the same query on my local database and it works fine.
Could there be a special setting in mysql that treats dot differently than it usually does?
user1084605, I tried to replicate the problem (using MySQL version 5.1.37), but got exactly the opposite results as you. See below:
mysql> create table test (username varchar(100));
Query OK, 0 rows affected (0.01 sec)
mysql> insert into test values ('marco.polo');
Query OK, 1 row affected (0.00 sec)
mysql> SELECT * FROM test WHERE `username`='marco.polo';
+------------+
| username |
+------------+
| marco.polo |
+------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM test WHERE `username` LIKE '%.polo%';
+------------+
| username |
+------------+
| marco.polo |
+------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM test WHERE `username` LIKE 'polo';
Empty set (0.00 sec)
According to the MySQL docs, the only special characters when using the LIKE operator are "%" (percent: matches 0, 1, or many characters) and "_" (underscore: matches one and only one character).
http://dev.mysql.com/doc/refman/5.0/en/string-comparison-functions.html
A "." (period) does have special meaning for MySQL's REGEXP operator, but it should still match a literal period in your column.
http://dev.mysql.com/doc/refman/5.0/en/regexp.html
Can you replicate the SQL statements I ran above and paste your results in reply?
As #cen already mentioned, character set can causes that problem.
I have had this sample:
`email` VARCHAR(45) CHARACTER SET 'armscii8' NOT NULL,
this is was in the .sql dump, which I receive.
So, when I was trying to fetch object with this email
I couldn't get it.
The below query takes care of the scenario when we have only DOT operator in the columns.
SELECT * FROM test WHERE `username` LIKE '%.%';
I am creating my first serious project in PHP and I want to make sure I have my database setup correctly. It is utf8_general_ci and for example the max I want usernames to be is 20 characters, so the username field in the database would be a varchar(20)? Sorry if this is stupid, it is just I read something somewhere that is making me question myself.
Yes you're right:
CREATE DATABASE my_test_db
DEFAULT CHARACTER SET utf8
DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
USE my_test_db;
Database changed
CREATE TABLE users (username varchar(20));
Query OK, 0 rows affected (0.04 sec)
INSERT INTO users VALUES ('abcdefghijklmnopqrstuvwxyz');
Query OK, 1 row affected, 1 warning (0.00 sec)
SELECT * FROM users;
+----------------------+
| username |
+----------------------+
| abcdefghijklmnopqrst |
+----------------------+
1 row in set (0.00 sec)