I have been researching this problem on StackOverflow for more than 24 hours and decided that this isn't already covered elsewhere even though there are many Q&A about the same topic.
I am using HeidiSQL 9.3 against MariaDB 10.1 and have a strange problem as follows: I originally accepted the default collation when I created my database then realized that this wasn't what I wanted and tried to change it with
ALTER DATABASE InternalFulfillment CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
This has no effect, and the database is still reported as ucs2_bin and all of the procedures and functions are ucs2_bin as well. I tried all of the advice from every Q&A I could find on StackOverflow including these statements:
SET collation_connection = 'utf8mb4_unicode_ci';
SET NAMES 'utf8mb4';
SET CHARACTER SET 'utf8mb4';
When I drop and recreate the procedures they still come back as ucs2_bin.
The strangest thing of all is that if I drop and recreate the database with the name 'InternalFulfilment' the collation is wrong, but if I create a database with a different name then I get the collation I want, and running the script that creates the stored procedures creates procedures with the utf8mb4_unicode_ci collation.
It seems like MariaDB and/or HeidiSQL is remembering the original collation that I used when I first created the 'InternalFulfillment' database, and always uses ucs2_bin collation whenever I create a database with this name.
Does anyone have any idea where this might be stored so I can clear it. Thanks.
Additional comments after reading answers below
After leaving this overnight, the next morning I was able to drop and recreate the database with a different collation, but now it is stuck on the new collation.
Following on from the answer from #Anse:
DROP DATABASE IF EXISTS `InternalFulfillment`;
CREATE DATABASE `InternalFulfillment` /*!40100 COLLATE 'ucs2_bin' */;
USE `InternalFulfillment`;
CREATE TABLE `table1` (
`column1` VARCHAR(50) NULL
)
COLLATE='ucs2_bin'
ENGINE=InnoDB;
DELIMITER //
CREATE DEFINER=`root`#`%` PROCEDURE `proc1`(IN `param1` VARCHAR(50))
DETERMINISTIC
BEGIN
SELECT
column1
FROM
table1 t
WHERE
t.column1 = param1;
END//
DELIMITER ;
CALL proc1('test');
Produces: /* SQL Error (1267): Illegal mix of collations (ucs2_bin,IMPLICIT) and (utf8mb4_general_ci,IMPLICIT) for operation '=' */. If I re-run this script with utf8mb4_general_ci then it completes without error.
Yesterday my database was stuck in ucs2_bin and today it is stuck in utf8mb4_general_ci so there is something that is cached with a fairly long expiry time.
Looks like there is some collation cache in MariaDB. I'm the author of HeidiSQL, and I'm pretty sure there is no such collation cache thing in HeidiSQL itself, so it must be a MySQL and/or MariaDB issue.
However, I just tried to reproduce the issue on a MySQL v5.7.9 server on my local Windows, without luck:
CREATE DATABASE `InternalFulfillment` /*!40100 COLLATE 'ucs2_bin' */;
CREATE TABLE `table1` (
`Column 1` VARCHAR(50) NULL
)
COLLATE='ucs2_bin'
ENGINE=InnoDB;
Both the database and table1 have ucs2_bin collation, as expected.
ALTER DATABASE `internalfulfillment` COLLATE 'utf8mb4_general_ci';
CREATE TABLE `table2` (
`Column 1` VARCHAR(50) NULL
)
COLLATE='utf8mb4_general_ci'
ENGINE=InnoDB;
Now, the database and the newly created table2 report the changed collation, as expected:
SELECT `DEFAULT_COLLATION_NAME` FROM `information_schema`.`SCHEMATA`
WHERE `SCHEMA_NAME`='internalfulfillment';
>> utf8mb4_general_ci
SELECT TABLE_NAME, TABLE_COLLATION FROM `information_schema`.`TABLES`
WHERE TABLE_SCHEMA='internalfulfillment';
TABLE_NAME | TABLE_COLLATION
table1 | ucs2_bin
table2 | utf8mb4_general_ci
So, my guess is that you have hit a bug in MariaDB.
Check these:
SHOW VARIABLES LIKE 'char%'; -- If anything say 'ucs2...', you should change it
SHOW CREATE DATABASE ...; -- The /*!40100 */ 'comment' is for hiding the clause for old versions.
SHOW CREATE PROCEDURE ...\G
Some basics (which you seem to already understand):
The CHARACTER SET and COLLATION on a database are only defaults for when you do CREATE TABLE without specifying them.
Similarly, the CHARACTER SET and COLLATION specified for a table are only defaults for for the columns.
To change the COLLATION for a given column(s), you need to use ALTER TABLE ... MODIFY COLUMN ....
To change the defaults, ALTER can be used on the database or table; but that only affects future tables or columns.
Another issue... The CHARACTER SET and COLLATION for any stored routine is defined when the routine is declared. To change either or both of those, you must DROP and reCREATE the routine.
Going forward, utf8mb4 is the main CHARACTER SET to use. ucs2 (and most of the other charsets) should almost never be used. If you can discover where "ucs2" originated, root it out.
Related
I tried using these two SQL statements to create a database:
CREATE DATABASE IF NOT EXISTS `dbname` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
and
CREATE DATABASE IF NOT EXISTS `dbname` DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_general_ci;
Both of these SQLs got the same result:
mysql> show create database `dbname`;
+----------------------+--------------------------------------------------------+
| Database | Create Database |
+----------------------+--------------------------------------------------------+
| dbname | CREATE DATABASE `dbname` /*!40100 DEFAULT CHARACTER SET utf8mb4 */ |
+----------------------+--------------------------------------------------------+
But, there isn't an option named CHARSET for creating database in MySQL documentation https://dev.mysql.com/doc/refman/8.0/en/create-database.html, I want to know the differences between these two statements.
I'm trying to find any official documentation about CHARSET too but all of the search result return link to CHARACTER SET. What I do find is this MariaDB documentation which have a CREATE TABLE query using CHARSET. So, I'm guessing CHARSET is a valid synonym to CHARACTER SET.
FYI: I notice that MySQL have a lot of similar functions but slightly - to moderately different name such as CURRENT_DATE and CURDATE(). Most of these functions have almost identical name only differs slightly in spelling. However, there are two functions that I know in particular have totally different names and noticeable difference in length but perform exactly the same operation. These functions are CURRENT_TIMESTAMP and NOW(). Here is a demo.
I was getting an error on inserting emoji through my android app in mysql db that :
java.sql.SQLException: Incorrect string value: '\xF0\x9F\x98\x8D\xF0\x9F...' for column 'question_text' at row 1
so i searched for it and found out that i had to Change the character set and collation properties of the databases, tables, and columns to use utf8mb4 instead of utf8.
Well i thought of applying to just one column first so i executed the query:
ALTER TABLE question MODIFY question_text varchar(100)
CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
But got the same error , than i thought of applying to the entire database so i executed the query:
ALTER DATABASE my_database CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
but still getting the same error . Can someone explain to me what i could be missing?
I am using MySQL workbench to design my MySQL schema and I need my database to be case-sensitive. I have set the default collation method to latin1_general_cs, latin1_bin, and utf8_bin to no avail. When I check the collation version in MySQL by using the command SELECT collation(version()), it returns
mysql> select collation(version());
+----------------------+
| collation(version()) |
+----------------------+
| utf8_general_ci |
+----------------------+
1 row in set (0.00 sec)
Regardless of what default method I have chosen.
When I do the following search:
mysql> select * from table_name where t_name = "search_request";
I get back SEARCH_REQUEST, as well as search_request.
However, if I use the command
mysql> select * from table_name where t_name = "search_request" collate utf8_bin;
I get the anticipated result, search_request.
By the way, in my .sql file, I see the following:
CREATE SCHEMA IF NOT EXISTS `database_name` DEFAULT CHARACTER SET utf8 COLLATE utf8_bin ;
USE `database_name` ;
which makes me incredibly confused why I am not seeing the correct results or the correct default collation method. Any ideas?
I found that the problem was that I wasn't dropping my database schema before reinitializing it, so the changes weren't propagated to the current database.
Simply adding this command to the start of my .sql script:
DROP SCHEMA IF EXISTS `database_name` ;
Fixed my problem and I was able to make case-sensitive queries!
I am using Django on Bluehost. I created a form for user generated input, but unicode inputs from this form fails to be stored or displayed of characters. So I did a SO and google search that I should change the Collate and Character set of my database. I run this sql
ALTER DATABASE learncon_pywithyou CHARACTER SET utf8 COLLATE utf8_unicode_ci;
from python27 manage.py dbshell, which initiated a mysql shell, what shows on screen is
Query OK, 1 row affected (0.00 sec).
So I assume the problem is solved, but it is not actually. This sql has not done anything, as I later find it in phpMyAdmin provided by Bluehost. All the Varchar fields of all the tables are still in lantin1_swedish_ci collate.
So assume that alter table should work instead. I run this on mysql
alter table mytable character set utf8 collate utf8_unicode_ci;
although on screen it shows Query OK. 4 rows affected, it actually did nothing either, the collate of those fields in mytable did not change at all.
So I finally manually change the fields in phpMyAdmin for mytable and this works, now I am able to insert in this table with unicode and also they display correctly, but I have around 20 tables of such, I don't want to change them one by one manually.
Do we at all have a simple and effective way of changing Collate of each field to store and display correct unicodes?
Changing collation at the database level sets the default for new objects - existing collations will not be changed.
Similarly, at a table level, only new columns (See comment at the bottom) are affected with this:
alter table mytable character set utf8 collate utf8_unicode_ci;
However, to convert the collation of existing columns, you need to add convert to:
alter table mytable convert to character set utf8 collate utf8_unicode_ci;
In addition to #StuartLC ,
For Changing All 20 tables charset and collation use below query, Here world is database name
SELECT
CONCAT("ALTER TABLE ",TABLE_SCHEMA , ".",TABLE_NAME," CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci") AS AlterSQL
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = "world";
The above will prepare all ALTER queries which you need to run.
Does MySQL treat the default character setting in a cascading type of way?
For example, I'm looking at a script that generates the full db and it starts off with a statement like this:
CREATE SCHEMA IF NOT EXISTS `xyz` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci ;
so does that imply that all tables that are created as part of this db will use the UTF8 charset by default? And if I wanted to use a different charset for a given table I would simply have to define it on the CREATE TABLE statement? Is that accurate? Can I override on a specific field of a specific table too? thanks!
Yes. If you set the charset and collation at the database (schema) level, it will apply to any newly created tables inside that database, unless those tables specify their own charset/collation in the CREATE TABLE statement.
And yes, you can also specify charset/collation on a per-field basis. Example from the MySQL manual:
CREATE TABLE t1
(
col1 VARCHAR(5)
CHARACTER SET latin1
COLLATE latin1_german1_ci
);