I have a simple table of a single column with rows of char(12) like:
DRF4482
DRF4497
DRF451
DRF4515
EHF452
FJF453
GKF4573
I want to select all of the rows that are between D and F, and have 4 numbers at the end. Like DRF4482, DRF4497, DRF4515, etc. I've tried a number of different wildcard combinations but I get no rows. I'm using:
SELECT * FROM `expired` WHERE id like '%[D-F][A-Z][A-Z]____';
I've even tried to broaden it to:
SELECT * FROM `expired` WHERE id like '%[D-F]%';
and that returns nothing as well.
I've even tried COLLATE latin1_bin based on some other posts but that didn't work either. My table is utf8, but I've created a second table as latin1 and tried a few different collations with the same results - no rows.
Where is my error?
You need to use REGEXP instead of LIKE. Notice that the syntax is a little different; it doesn't do anything with the SQLish % wildcard characters.
So, you want
id REGEXP '[D-F][A-Z][A-Z][0-9]{4}'
for this app. Hopefully you don't have multibyte characters in these strings, because MySQL's regexp doesn't work correctly in those circumstances.
Related
It states in the docs:
utf8_unicode_ci supports mappings such as expansions; that is, when one character compares as equal to combinations of other characters. For example, in German and some other languages “ß” is equal to “ss”.
I tried this out with an utf8mb4_unicode_ci collated table, with a row that has the name Heß.
select * from users where name = "Hess"
return the row with Heß. However, when I do the same with the like syntax
select * from users where name LIKE "%Hess%"
it returns no match.
Why does this happen and how does one fix it?
Note that this question is NOT about searching for (non)accented characters.
Suppose I have a table where there is a column name, with collation utf8mb4_unicode_ci.
This collation works perfectly for the purpose of selecting the base selection
in a case-insensitive, accent-insensitive way.
The problem is that I need to order the results in an accent-sensitive and case-insensitive way.
The purpose of this is to select every name starting with some character/string and sort them "alphabetically", first should be not-accented, then accented.
From selection e.g.:
Črpw
Cewo
céag
čefw
The final results should be:
Cewo
céag -- because accented e is more than non-accented
čefw
Črpw -- because r is more than e
Note that c/C < č/Č , but lower/upper cases are handled as equals.
I tried searching for this problem, but there are only popping similar questions or questions about searching, which is not the case, the searching itself is fine.
From mentioned I've tried this test query:
SELECT * FROM
(SELECT 'Črpw' as t
UNION SELECT 'Cewo'
UNION SELECT 'céag'
UNION SELECT 'čefw')virtual
ORDER BY t COLLATE utf8mb4_czech_ci ASC
Which produces something very similar to what I want
céag
Cewo
čefw
Črpw
But note that é gets ordered before e.
Is there a way how to get to the results order I want to have?
Using: MySQL 5.5.54 (Debian)
I have a table in MySQL that contains people's names and now some people are putting in characters with accents. Whatever original character set/collation this database used, it could not handle characters with accents and characters. Characters such as 'é' turned into 'é', characters such as 'ü' turned into 'ü' in the front-end applications that connect to this database. The database has since been modified (as well as most front-end applications that needed it) so that these fields use a UTF8 encoding and collation. Most names with accents now render correctly.
The problem is, there are some fields that now have a literal 'é' (some weird double-encoding caused by manually copying and pasting stuff? I don't know, I wasn't there!). I now need to find all of these and modify these fields to use the correct accented characters. To find them, I wrote a query (based on the table I found here):
select count(*), bad_char
from some_table
inner join (
select '€' as bad_char union
select '‚' as bad_char union
select 'Æ’' as bad_char union
select '„' as bad_char union
select '…' as bad_char union
...
-- snip a whole bunch
...
select 'þ' as bad_char union
select 'ÿ' as bad_char ) bad_chars
where some_table.some_text_field like CONCAT('%',bad_chars.bad_char,'%')
group by bad_char
order by count(*);
And I get results like this:
count(*), bad_char
------------------
'1', '¯'
'1', 'Ñ'
'1', 'Ö'
'1', 'Ž'
'1', 'Ç'
'1', '¬'
...snip...
'1797', 'ß'
'4450', 'Ê'
But I have a feeling this doesn't quite work the way I think it does. One of the "bad_char" combinations I'm searching for is 'Ê' but when I run a query like this:
select some_text_field from some_table where some_text_field like '%Ê%';
I get many results that are 'as' which are the same letters but without the accents. But for other cases such as '¯' the query seems to work fine.
How can I get this query to not treat 'as' the same as 'Ê'?
I get many results that are 'as' which are the same letters but without the accents.
That would be an issue of the collation used - those are rule sets for character comparison, and they define which characters are to be treated as equal in different languages.
But you can use the BINARYoperator to change that directly within the query.
For Mojibake ('é' turned into 'é'), the fix for the data is
ALTER TABLE Tbl MODIFY COLUMN col VARBINARY(...) ...;
ALTER TABLE Tbl MODIFY COLUMN col VARCHAR(...) ... CHARACTER SET utf8 ...;
Your SELECTs sound like Mojibake -- € should have been €, correct?
The fix for the code is to understand that
The bytes you have in the client are correctly encoded in utf8 (good).
You connected with SET NAMES latin1 (or set_charset('latin1') or ...), probably by default. (It should have been utf8.)
The column in the tables may or may not have been CHARACTER SET utf8, but it should have been that.
The SET NAMES is often done by language-specific code; what programming language are you using.
Use SHOW CREATE TABLE to see what CHARACTER SET you are using.
For anyone else trying to fix broken UTF8 accented text on latin1 fields, when some of them have proper latin1 and others have broken UTF8, the solution is this:
update posts set post_text = #txt where length(post_text) = length(#txt := convert(binary post_text using utf8));
This will fix only those rows with wrongly encoded UTF8 back into proper latin1 encoding, and á will become á, etc.
I need to select columns that end in an uppercase extension.
for example, look at the following table.
id - picture
1 - abc.JPG
2 - def.jpg
3 - 123.jpg
4 - xyz.JPG
the results should give me the rows 1 and 4 because JPG are in uppercase.
can anyone help?
I'm far from an expert, but case sensitivity has hung me up before. Are you able to modify the structure of your table? One thing that may help is changing the collation of the table as follows (SQLFiddle here):
CREATE TABLE pics (id INT, picture VARCHAR(200))
CHARACTER SET latin1 COLLATE latin1_general_cs;
INSERT INTO pics VALUES
(1, 'abc.JPG'),
(2, 'def.jpg'),
(3, '123.jpg'),
(4, 'xyz.JPG')
The _cs stands for case sensitive, and I believe the default is case insensitive, which makes case-based comparisons a bit trickier. You can then use the following query to get your rows:
SELECT *
FROM pics
WHERE picture REGEXP '\.[[:upper:]+]$'
If you do not have access to your underlying table, you could try the following, which casts the column in a different character set (latin1), and then changes the collation to support case-insensitive comparisons (SQLFiddle here):
SELECT *
FROM pics
WHERE CAST(picture AS CHAR CHARACTER SET latin1)
COLLATE latin1_general_cs REGEXP '\.[[:upper:]+]$'
Some REGEXPs:
'[.][[:upper:]]+$' -- just uppercase letters
'[.].*[[:upper:]]' -- at least one uppercase letter
'[.].*[[:lower:]]' -- at least one lowercase letter; AND together with previous to get upper and lower, etc.
If there could be two "." in the filename, then consider using
SUBSTRING_INDEX(picture, '.', -1)
to isolate the 'extension'.
Most SQL languages have a UCASE or UPPER function to convert text to uppercase. I'm also taking advantage of the RIGHT function which isn't in all SQL dialects. If your SQL doesn't have a RIGHT function you'll have to futz with SUBSTRING and LENGTH to get the right three characters in picture.
Select id, picture
from table
where UPPER(RIGHT(TRIM(picture),3)) = RIGHT(TRIM(picture),3)
If the same text converted to uppercase is the same as the unconverted text then it is uppercase in the database and will be selected.
As it can not always be assumed that the file extension will be 3 letters you may use the following to get the first chart after the period and compare it to see if it is uppercase:
select * from table where SUBSTRING(picture,CHARINDEX('.',picture) + 1,1)
= upper(SUBSTRING(picture,CHARINDEX('.',picture) + 1,1)) collate SQL_Latin1_General_CP1_CS_AS
I have a database that stores a large number of codes, these codes are used to validate submission of a form. When ever i run the following query i get zero rows back
SELECT * FROM `codes` WHERE `voucher` = 'JTBLYNQ9HA'
but when i run the following query it bring back the single row with the code in it.
SELECT * FROM `codes` WHERE `voucher` LIKE CONVERT( _utf8 '%JTBLYNQ9HA%' USING latin1 ) COLLATE latin1_swedish_ci LIMIT 0 , 30
What am i doing wrong which causes the first query to fail or should is it best practise to use the second query?
Thanks for the help
The two queries are not equivalent. The first one is looking for a code whose voucher is exactly "JTBLYNQ9HA", the second one is looking for a code whose voucher contains that string (for instance, "ABCDEFGJTBLYNQ9HAHIJKLM").
The character set conversion and COLLATE are almost certainly irrelevant.