MySQL convert to UTF8 without structure change - mysql

Question

I have a rather large database that I am trying to convert from charset and collation latin1/latin1_swedish_ci to utf8mb4/utf8mb4_unicode_ci. I am hoping to setup replication to a slave, run the conversion, and then promote the slave when finished as to avoid downtime.
I noticed that when running the query...
ALTER TABLE `sometable` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
...MySQL automatically converts text to mediumtext or mediumtext to longtext, etc.
Is there a way to turn this feature off? It is nice that MySQL has this feature, but the problem is that it breaks replication because the structure of the tables on the slave is different from master.

Answer 1

As documented under ALTER TABLE Syntax:
For a column that has a data type of VARCHAR or one of the TEXT types, CONVERT TO CHARACTER SET will change the data type as necessary to ensure that the new column is long enough to store as many characters as the original column. For example, a TEXT column has two length bytes, which store the byte-length of values in the column, up to a maximum of 65,535. For a latin1 TEXT column, each character requires a single byte, so the column can store up to 65,535 characters. If the column is converted to utf8, each character might require up to three bytes, for a maximum possible length of 3 × 65,535 = 196,605 bytes. That length will not fit in a TEXT column's length bytes, so MySQL will convert the data type to MEDIUMTEXT, which is the smallest string type for which the length bytes can record a value of 196,605. Similarly, a VARCHAR column might be converted to MEDIUMTEXT.
To avoid data type changes of the type just described, do not use CONVERT TO CHARACTER SET. Instead, use MODIFY to change individual columns. For example:
ALTER TABLE t MODIFY latin1_text_col TEXT CHARACTER SET utf8;
ALTER TABLE t MODIFY latin1_varchar_col VARCHAR(M) CHARACTER SET utf8;

Answer 2

(Not really an answer, but some illustrative examples.)
Case 1: Text is correctly stored as latin1 in latin1 column; use CONVERT TO
mysql> CREATE TABLE alters (
-> c VARCHAR(11) CHARACTER SET latin1 NOT NULL
-> );
mysql> INSERT INTO alters (c) VALUES ('aabc'), (UNHEX('61e06263')), (UNHEX('61e16263'));
mysql> SELECT c, HEX(c) from alters;
+-------+----------+
| c | HEX(c) |
+-------+----------+
| aabc | 61616263 |
| aàbc | 61E06263 |
| aábc | 61E16263 |
+-------+----------+
mysql> ALTER TABLE alters CONVERT TO CHARACTER SET utf8;
mysql> SELECT c, HEX(c) from alters;
+-------+------------+
| c | HEX(c) |
+-------+------------+
| aabc | 61616263 |
| aàbc | 61C3A06263 |
| aábc | 61C3A16263 |
+-------+------------+
mysql> -- Observation: text was correctly converted to utf8.
mysql> SHOW CREATE TABLE alters\G
Create Table: CREATE TABLE `alters` (
`c` varchar(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Case 2: Text is correctly stored as latin1 in latin1 column; use "Double ALTER"
mysql> CREATE TABLE alters (
-> c VARCHAR(11) CHARACTER SET latin1 NOT NULL
-> );
mysql> INSERT INTO alters (c) VALUES ('aabc'), (UNHEX('61e06263')), (UNHEX('61e16263'));
mysql> ALTER TABLE alters MODIFY c VARBINARY(11) NOT NULL;
mysql> ALTER TABLE alters MODIFY c VARCHAR(11) CHARACTER SET utf8 NOT NULL;
Query OK, 3 rows affected, 2 warnings (0.10 sec)
Records: 3 Duplicates: 0 Warnings: 2
mysql> SHOW WARNINGS;
+---------+------+----------------------------------------------------------+
| Level | Code | Message |
+---------+------+----------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xE0bc' for column 'c' at row 2 |
| Warning | 1366 | Incorrect string value: '\xE1bc' for column 'c' at row 3 |
+---------+------+----------------------------------------------------------+
mysql> SELECT c, HEX(c) from alters;
+------+----------+
| c | HEX(c) |
+------+----------+
| aabc | 61616263 |
| a | 61 |
| a | 61 |
+------+----------+
mysql> -- Observation: text was truncated ! BAD
mysql> SHOW CREATE TABLE alters\G
Create Table: CREATE TABLE `alters` (
`c` varchar(11) CHARACTER SET utf8 NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Case 3: Text was incorrectly stored as utf8 in a latin1 column; use the "Double ALTER to fix it
mysql> CREATE TABLE alters (
-> c VARCHAR(11) CHARACTER SET latin1 NOT NULL
-> );
mysql> INSERT INTO alters (c) VALUES ('aabc'), (UNHEX('61c3a06263')), (UNHEX('61c3a16263'));
mysql> ALTER TABLE alters MODIFY c VARBINARY(11) NOT NULL;
mysql> ALTER TABLE alters MODIFY c VARCHAR(11) CHARACTER SET utf8 NOT NULL;
mysql> SELECT c, HEX(c) from alters;
+-------+------------+
| c | HEX(c) |
+-------+------------+
| aabc | 61616263 |
| aàbc | 61C3A06263 |
| aábc | 61C3A16263 |
+-------+------------+
mysql> SHOW CREATE TABLE alters\G
Create Table: CREATE TABLE `alters` (
`c` varchar(11) CHARACTER SET utf8 NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Case 4: Using ALTER ... MODIFY; note the LENGTH and CHAR_LENGTH
mysql> CREATE TABLE alters (
-> c VARCHAR(9) CHARACTER SET latin1 NOT NULL
-> );
mysql> INSERT INTO alters (c) VALUES ('aabc'), (UNHEX('61e06263')),
-> (UNHEX('61e16263')),
-> (UNHEX('61e162633536373839'));
mysql> SELECT c, HEX(c), LENGTH(c), CHAR_LENGTH(c) from alters;
+------------+--------------------+-----------+----------------+
| c | HEX(c) | LENGTH(c) | CHAR_LENGTH(c) |
+------------+--------------------+-----------+----------------+
| aabc | 61616263 | 4 | 4 |
| aàbc | 61E06263 | 4 | 4 |
| aábc | 61E16263 | 4 | 4 |
| aábc56789 | 61E162633536373839 | 9 | 9 |
+------------+--------------------+-----------+----------------+
mysql> ALTER TABLE alters MODIFY c VARCHAR(9) CHARACTER SET utf8 NOT NULL;
mysql> SELECT c, HEX(c), LENGTH(c), CHAR_LENGTH(c) from alters;
+------------+----------------------+-----------+----------------+
| c | HEX(c) | LENGTH(c) | CHAR_LENGTH(c) |
+------------+----------------------+-----------+----------------+
| aabc | 61616263 | 4 | 4 |
| aàbc | 61C3A06263 | 5 | 4 |
| aábc | 61C3A16263 | 5 | 4 |
| aábc56789 | 61C3A162633536373839 | 10 | 9 |
+------------+----------------------+-----------+----------------+
mysql> SHOW CREATE TABLE alters\G
Create Table: CREATE TABLE `alters` (
`c` varchar(9) CHARACTER SET utf8 NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
Notes:
No Warnings except for the one case where I did SHOW.
Default table CHARSET was not changed, but that is not a problem.