I have an existing table that holds valid JSON data but is stored as LONGTEXT (character set utf8mb4 with collation utf8mb4_bin). Doing JSON queries on this is highly inefficient and I want to change the data type of this column to JSON.
When I do so in HeidiSQL I get an: 'COLLATION 'utf8mb4_bin' is not valid for CHARACTER SET 'binary'. I can fix this by resetting the collation to empty.
I mention this because I thought the character set for a JSON column is utf8mb4 and it's default collation is utf8mb4_bin. In the JSON are some GUIDs and when I try to query them, it appears that I have to use LIKE. If I use the 'normal' = I get no results.
Works: SELECT * FROM MyTable WHERE Data->>"$.Shop[*].ContactPerson.UserId" LIKE "%b1b9ad95-1098-4e6c-a697-50c2a47dc301%";
Doesn't work: SELECT * FROM MyTable WHERE Data->>"$.Shop[*].ContactPerson.UserId" = "b1b9ad95-1098-4e6c-a697-50c2a47dc301";
Is that a problem with my syntax or is it related to collation (I'd say no, but it's the only discrepancy I find). BTW: Leaving out the % in the LIKE also produces no results.
Create/insert:
CREATE TABLE Temp ( `Data` LONGTEXT COLLATE UTF8MB4_BIN );
INSERT INTO Temp( `Data` ) VALUES ("{\"Shop\":[{\"ContactPerson\": {\"UserId\":\"b1b9ad95-1098-4e6c-a697-50c2a47dc301\"}},{\"ContactPerson\": {\"UserId\":\"B27DA5A7-D678-4513-8A44-BD76CC4651CC\"}}]}");
INSERT INTO Temp( `Data` ) VALUES ("{\"Shop\":[{\"ContactPerson\": {\"UserId\":\"82899A81-2024-4F68-917A-710764296A21\"}},{\"ContactPerson\": {\"UserId\":\"AE59DCA7-32AF-4131-93C7-A1BB698DF8E0\"}}]}");
INSERT INTO Temp( `Data` ) VALUES ("{\"Shop\":[{\"ContactPerson\": {\"UserId\":\"4154477B-1B70-4F25-9E4B-2CFBBF4F678F\"}},{\"ContactPerson\": {\"UserId\":\"B27DA5A7-D678-4513-8A44-BD76CC4651CC\"}}]}");
Trying the update: ALTER TABLE `Temp` MODIFY `Data` JSON;
And now it works... (of course) :(
So apparently when using the table-editor in HeidiSQL it fails (I've retried this). Using a SQL statement actually does what I want.
That still leaves me with having to use the LIKE as opposed to = in my query.
Thank you to #Rick James for the "aha moment": Where clause for JSON field on array of objects
So in my case:
SELECT * FROM `Temp` WHERE JSON_SEARCH(`Data`, 'one', 'b1b9ad95-1098-4e6c-a697-50c2a47dc301', NULL, '$.Shop[*].ContactPerson.UserId') IS NOT NULL;
Update:
Issue was reported and will be fixed in next build (https://github.com/HeidiSQL/HeidiSQL/issues/1652)
I'm using MySQL 8.0.4 (rc4) I need MySQL 8 because it's the only version of MySQL that supports CTEs.
My database is created thus:
CREATE DATABASE IF NOT EXISTS TestDB
DEFAULT CHARACTER SET utf8mb4
DEFAULT COLLATE utf8mb4_general_ci;
USE TestDB;
SET sql_mode = 'STRICT_TRANS_TABLES';
CREATE TABLE IF NOT EXISTS MyTable (
(...)
Body LONGBLOB NOT NULL,
(...)
);
When I try to insert raw byte data to this description field, I receive this error:
Error 1366: Incorrect string value: '\x8B\x08\x00\x00\x00\x00...' for column 'Body' at row 1.
This is the insert statement I'm using.
REPLACE INTO MyTable
SELECT Candidate.* FROM
(SELECT :Id AS Id,
(...)
:Body AS Body,
(...)
) AS Candidate
LEFT JOIN MyTable ON Candidate.Id = MyTable.Id
WHERE (
(...)
);
How could there be an incorrect string value for BLOB? Doesn't BLOB mean I can insert quite literally anything?
What's the : stuff? Why have the nested query? May we see actual SQL? What language are you using? It sounds like the "binding" tried to apply character set rules, when it should not. May we see the code that did the substitution of the : stuff?
BLOBs have not character set. As long as you can get the bytes past the parser, there should be no problem.
However, I find this to be a better way to do it...
In the app language, generate a hex string, then use that in
INSERT INTO ... VALUES (..., UNHEX(the-hex-string), ...)
I am getting the following error message.
Cannot resolve the collation conflict between "Latin1_General_CI_AI" and "SQL_Latin1_General_CP1_CI_AS" in the equal to operation.
I only get it when I place this code below in my WHERE clause.
WHERE Region IN (SELECT Token FROM dbo.getParmsFromString(#Region))
Now #Region contains all the values from my multi-select fields from SSRS.
Below is the code for the function that is used.
CREATE FUNCTION [dbo].[getParmsFromString]
(#String VARCHAR(MAX))
RETURNS #Parms TABLE
(
Token VARCHAR(MAX)
)
AS
BEGIN
IF CHARINDEX(',', #String) != 0
BEGIN
;WITH cte0(Token, List) AS
(
SELECT SUBSTRING(#String, 1, CHARINDEX(',',#String,1) - 1)
,SUBSTRING(#String,CHARINDEX(',',#String,1) + 1, LEN(#String)) + ','
UNION ALL
SELECT SUBSTRING(List,1,ISNULL(CHARINDEX(',',List,1) - 1,1))
,SUBSTRING(List,CHARINDEX(',',List,1) + 1, LEN(List))
FROM cte0
WHERE LEN(cte0.List) > 0
)
INSERT INTO #Parms (Token)
SELECT Token
FROM cte0
OPTION (MAXRECURSION 0)
RETURN;
END
ELSE
INSERT INTO #Parms
SELECT #String
RETURN;
END
Try changing
RETURNS #Parms TABLE
(
Token VARCHAR(MAX)
)
with
try changing RETURNS #Parms TABLE
(
Token VARCHAR(MAX) COLLATE DATABASE_DEFAULT
)
and
WHERE Region IN (SELECT Token FROM dbo.getParmsFromString(#Region))
with
WHERE Region COLLATE DATABASE_DEFAULT IN (SELECT Token FROM dbo.getParmsFromString(#Region))
Generally this type of error occurs when you try to compare the the data of different regions or when you compare data using a specific encryption with other data using a different encryption.
The most probable reason is that their tempdb is using the collation "SQL_Latin1_General_CP1_CI_AS" while the database is using "Latin1_General_CI_AS". As a result, temp objects are created under the collation "SQL_Latin1_General_CP1_CI_AS" and then fail to compare with database objects of the database which are using the collation "Latin1_General_CI_AS".
The easiest fix and also the one which would recommended would be to run the database on a server which was installed using collation "Latin1_General_CI_AS".
FYI. SQL collations("SQL_Latin1_General_CP1_CI_AS") are present in sql server for backward compatibility. When dealing with international data or databases using unicode and non unicode data, it is recommended to use windows collations ("Latin1_General_CI_AS").
You can change your database collation by:
use master
ALTER DATABASE "Your database"
COLLATE Latin1_General_CI_AS;
SELECT name, collation_name
FROM sys.databases;
and if needed you can also change the collation of "master" database i.e. rebuilding the database, for this go through to these links:
http://msdn.microsoft.com/en-us/library/dd207003(v=sql.100).aspx
http://sqlbuzz.wordpress.com/2011/08/20/how-to-rebuild-master-database-aka-rebuilding-sql-server-2008r2/
but make sure you backup all your database before doing this.
I am using char_length() to measure the size of "Русский": strangely, instead of telling me that it's 7 chars, it tells me there are 14. Interestingly if the query is simply...
SELECT CHAR_LENGTH('Русский')
...the answer is correct. However if I query the DB instead, the anser is 14:
SELECT CHAR_LENGTH(text) FROM locales WHERE lang = 'ru-RU' AND name = 'lang_name'
Anybody go any ideas what I might be doing wrong? I can confirm that the collation is utf8_general_ci and the table is MyISAM
Thanks,
Adrien
EDIT: My end objective is to be able to measure the lengths of records in a table containing single and double-byte chracters (eg. English & Russian, but not limited to these two languages only)
Because of two bytes is used for each UTF8 char.
See http://dev.mysql.com/doc/refman/5.5/en/string-functions.html#function_char-length
mysql> set names utf8;
mysql> SELECT CHAR_LENGTH('Русский'); result - 7
mysql> SELECT CHAR_LENGTH('test'); result - 4
create table test123 (
text VARCHAR(255) NOT NULL DEFAULT '',
text_text TEXT) Engine=Innodb default charset=UTF8;
insert into test123 VALUES('русский','test русский');
SELECT CHAR_LENGTH(text),CHAR_LENGTH(text_text) from test123; result - 7 and 12
I have tested work with: set names koi8r; create table and so on and got invalid result.
So the solution is recreate table and insert all data after setting set names UTF8.
the function return it's anwser guided by the most adjacent charset avaiable
in the case of a column, the column definition
in the case of a literal, the connection default
review the column charset with:
SELECT CHARACTER_SET_NAME FROM information_schema.`COLUMNS`
where table_name = 'locales'
and column_name = 'text'
be careful, it is not filtered by table_schema
Just got this answer from a previous question and it works a treat!
SELECT username, (SUM(rating)/COUNT(*)) as TheAverage, Count(*) as TheCount
FROM ratings WHERE month='Aug' GROUP BY username HAVING TheCount > 4
ORDER BY TheAverage DESC, TheCount DESC
But when I stick this extra bit in it gives this error:
Documentation #1267 - Illegal mix of
collations
(latin1_swedish_ci,IMPLICIT) and
(latin1_general_ci,IMPLICIT) for
operation '='
SELECT username, (SUM(rating)/COUNT(*)) as TheAverage, Count(*) as TheCount FROM
ratings WHERE month='Aug'
**AND username IN (SELECT username FROM users WHERE gender =1)**
GROUP BY username HAVING TheCount > 4 ORDER BY TheAverage DESC, TheCount DESC
The table is:
id, username, rating, month
Here's how to check which columns are the wrong collation:
SELECT table_schema, table_name, column_name, character_set_name, collation_name
FROM information_schema.columns
WHERE collation_name = 'latin1_general_ci'
ORDER BY table_schema, table_name,ordinal_position;
And here's the query to fix it:
ALTER TABLE tbl_name CONVERT TO CHARACTER SET latin1 COLLATE 'latin1_swedish_ci';
Link
Check the collation type of each table, and make sure that they have the same collation.
After that check also the collation type of each table field that you have use in operation.
I had encountered the same error, and that tricks works on me.
[MySQL]
In these (very rare) cases:
two tables that really need different collation types
values not coming from a table, but from an explicit enumeration, for instance:
SELECT 1 AS numbers UNION ALL SELECT 2 UNION ALL SELECT 3
you can compare the values between the different tables by using CAST or CONVERT:
CAST('my text' AS CHAR CHARACTER SET utf8)
CONVERT('my text' USING utf8)
See CONVERT and CAST documentation on MySQL website.
I was getting this same error on PhpMyadmin and did the solution indicated here which worked for me
ALTER TABLE table CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci
Illegal mix of collations MySQL Error
Also I would recommend going with General instead of swedish since that one is default and not to use the language unless your application is using Swedish.
I think you should convert to utf8
--set utf8 for connection
SET collation_connection = 'utf8_general_ci'
--change CHARACTER SET of DB to utf8
ALTER DATABASE dbName CHARACTER SET utf8 COLLATE utf8_general_ci
--change CHARACTER SET of table to utf8
ALTER TABLE tableName CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci
I also got same error, but in my case main problem was in where condition the parameter that i'm checking was having some unknown hidden character (+%A0)
When A0 convert I got 160 but 160 was out of the range of the character that db knows, that's why database cannot recognize it as character other thing is my table column is varchar
the solution that I did was I checked there is some characters like that and remove those before run the sql command
ex:- preg_replace('/\D/', '', $myParameter);
Check that your users.gender column is an INTEGER.
Try: alter table users convert to character set latin1 collate latin1_swedish_ci;
You need to change each column Collation from latin1_general_ci to latin1_swedish_ci
I got this same error inside a stored procedure, in the where clause. i discovered that the problem ocurred with a local declared variable, previously loaded by the same table/column.
I resolved it casting the data to single char type.
In short, this error is caused by MySQL trying to do an operation on two things which have different collation settings. If you make the settings match, the error will go away. Of course, you need to choose the right setting for your database, depending on what it is going to be used for.
Here's some good advice on choosing between two very common utf8 collations: What's the difference between utf8_general_ci and utf8_unicode_ci
If you are using phpMyAdmin you can do this systematically by working through the tables mentioned in your error message, and checking the collation type for each column. First you should check which is the overall collation setting for your database - phpMyAdmin can tell you this and change it if necessary. But each column in each table can have its own setting. Normally you will want all these to match.
In a small database this is easy enough to do by hand, and in any case if you read the error message in full it will usually point you to the right place. Don't forget to look at the 'structure' settings for columns with subtables in as well. When you find a collation that does not match you can change it using phpMyAdmin directly, no need to use the query window. Then try your operation again. If the error persists, keep looking!
The problem here mainly, just Cast the field like this cast(field as varchar) or cast(fields as date)
I had this problem not because I'm storing in different collations, but because my column type is JSON, which is binary.
Fixed it like this:
select table.field COLLATE utf8mb4_0900_ai_ci AS fieldName
Use ascii_bin where ever possible, it will match up with almost any collation.
A username seldom accepts special characters anyway.
If you want to avoid changing syntax to solve this problem, try this:
Update your MySQL to version 5.5 or greater.
This resolved the problem for me.
I have the same problem with collection warning for a field that is set from 0 to 1. All columns collections was the same. We try to change collections again but nothing fix this issue.
At the end we update the field to NULL and after that we update to 1 and this overcomes the collection problem.
Was getting Illegal mix of collations while creating a category in Bagisto. Running these commands (thank you #Quy Le) solved the issue for me:
--set utf8 for connection
SET collation_connection = 'utf8_general_ci'
--change CHARACTER SET of DB to utf8
ALTER DATABASE dbName CHARACTER SET utf8 COLLATE utf8_general_ci
--change category tables
ALTER TABLE categories CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci
ALTER TABLE category_translations CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci
In my case it was something strange. I read an api key from a file and then I send it to the server where a SQL query is made. The problem was the BOM character that the Windows notepad left, it was causing the error that says:
SQLSTATE[HY000]: General error: 1267 Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='
I just removed it and everything worked like a charm
You need to set 'utf8' for all parameters in each Function. It's my case:
SELECT username, AVG(rating) as TheAverage, COUNT(*) as TheCount
FROM ratings
WHERE month='Aug'
AND username COLLATE latin1_general_ci IN
(
SELECT username
FROM users
WHERE gender = 1
)
GROUP BY
username
HAVING
TheCount > 4
ORDER BY
TheAverage DESC, TheCount DESC;
Make sure your version of MySQL supports subqueries (4.1+). Next, you could try rewriting your query to something like this:
SELECT ratings.username, (SUM(rating)/COUNT(*)) as TheAverage, Count(*) as TheCount FROM ratings, users
WHERE ratings.month='Aug' and ratings.username = users.username
AND users.gender = 1
GROUP BY ratings.username
HAVING TheCount > 4 ORDER BY TheAverage DESC, TheCount DESC