How to convert a big MySQL Database from utf8 to utf8mb4? - mysql

Question

I have a MySQL server (MariaDB Server version 10.5 running on Debian bullseye) with a 600 GB database. Due to compatibility issues I have to switch from UTF-8 to UTF8MB4.
I've found a few things about it, but I'm still unsure of the best way to do it. Since this is a productive system and a rollback is only possible on the testing system, I'm concerned about the data integrity and fear that difficulties may arise afterwards, for example with regard to performance.
What is the best and safest way to convert the database?
Is there anything special to consider?
Thanks for suggestions.

Answer 1

This article covers this process well. https://mathiasbynens.be/notes/mysql-utf8mb4
TL;DR;
So the biggest concern about this whole process is data integrity. What we mean by this is that we want our data not to get lost. So as a first step I would suggest creating a backup of the database. As soon as you have this copy then if something goes south then you can always have the copy to cover you. Safety first! So let's breakdown the process.
Create a backup of the database
Upgrade the MySQL server at least to v5.5.3 because this is the version that the UTF8MB4 started being supported
Modify the database, tables and columns. You can do this like
# For each database:
ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
# For each table:
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
# For each column:
ALTER TABLE table_name CHANGE column_name column_name VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
# The above line is just an example for a `VARCHAR` column.)
Now since mysql has kept this feature backwards compatible then you shouldn't be seeing any weird behaviour (data loss).
Check the maximum length of columns and index keys. When converting from utf8 to utf8mb4, the maximum length of a column or index key is unchanged in terms of bytes. So if you have a tinytext the maximum number of data that is able to store is 255 bytes. So with the increasing overhead you will be able to store less data. So in general check especially check your tinyx fields(you should check all of them but the tiny are more likely to run out of bytes).
Modify connection, client, and server character sets. Wherever you have utf8 in general you should replace with utf8mb4.
Run mysqlcheck -u root -p --auto-repair --optimize --all-databases in order to avoid running into some weird bugs where UPDATE statements didn’t have any effect, even though no errors were thrown.