How can I find the Non keyboard characters in MySQL? - mysql

Question

Related question to How can I find non-ASCII characters in MySQL?.
I want to check for col1 and col2 in my table given below for the cases where non keyboard characters are present.
+------------+----------+
| col1 | col2 |
+------------+----------+
| rewweew\s | 4rtrt |
| é | é |
| 123/ | h|h |
| ëû | û |
| ¼ | ¼ |
| *&^ | *%$ |
| #$ | ~!` |
+------------+----------+
My desired result will look like
+--------+-------+
| é | é |
| ëû | û |
| ¼ | ¼ |
+--------+-------+
In my case all characters present in English keyboard are allowed, i have to only find out row which have character not present in English keyboard like Chinese character etc.
I got the below mentioned query from the link How can I find non-ASCII characters in MySQL?
SELECT * FROM tableName WHERE NOT columnToCheck REGEXP '[A-Za-z0-9.,-]';
But its not working because character ~`#!#$%^&*()_-+=|}]{[':;?/>.<, are also allowed but its neglecting them.

Answer 1

This may be worth a try.
SELECT whatever
FROM tableName
WHERE columnToCheck <> CONVERT(columnToCheck USING ASCII)
The CONVERT(col USING charset) function will turns the unconvertable characters into
replacement characters. Then, the converted and unconverted text will be unequal.
Of course it's based on what is and isn't in the ASCII character repertoire, not what's on a particular keyboard. But it should probably do the trick for you. See this for more discussion.
http://dev.mysql.com/doc/refman/5.0/en/charset-repertoire.html
You can use any character set name you wish in place of ASCII. For example, if you want to find out which characters won't render correctly in code page 1257 (Lithuanian, Latvian, Estonian) use CONVERT(columnToCheck USING cp1257)
Edit
Your comment mentioned that you need also to detect some characters that are in the ASCII character set. I think you're asking about the so-called control characters, which have values from 0x00 to 0x1f, and then 0x7f. #Joni Salonen's approach helps get us there, but we need to do it in a way that's multibyte-character safe.
SELECT whatever
FROM tableName
WHERE CONVERT(columnToCheck USING ASCII) <> columnToCheck
OR CONVERT(columnToCheck USING ASCII) RLIKE '[[.NUL.]-[.US.][.DEL.]]'
If you look at http://www.asciitable.com/, you'll see that the OR clause here detects characters in the first column of the ASCII table, and the last character in the fourth column.

Answer 2

This query will return the rows that have characters outside of the ASCII range 0 - 127:
SELECT * FROM tableName WHERE NOT columnToCheck REGEXP '^[[.NUL.]-[.DEL.]]*$'
By English keyboard do you mean American or UK keyboard? The UK keyboard includes some non-ASCII characters, like the sterling pound symbol. If you want to accept those too you have to add them to the regular expression.