MySQL's UTF-8 character support - mysql

Question

I am getting this exception
java.sql.SQLException: Incorrect string value: '\xC2\x99 Adm...' for column when trying to insert a value in MySQL table. I found that \xC2\x99 maps to U+0099 (or \u0099) which is a 2-Byte character. From documentation, a character with 3 Bytes or less is supported by MySQL's UTF-8. I also read about utfmb4, but since this character is 2 Bytes and still giving this error, so the issue might be something else. Please suggest.

Answer 1

Looks like you are using the default collation latin1_swedish_ci whereas you should be using utf8_general_ci since you meant to store UTF-8 data in the column. Check MySQL Documentation on Character Sets and Collations in MySQL
You can use an ALTER command to change the collation character set
ALTER TABLE your_table CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;

Answer 2

Character U+0099 (aka '<control>') can indeed be encoded in Latin-1 aka ISO-8859-1 (more specifically, it's 0x99) and your connection appears to be properly configured to use UTF-8.
I suspect the problem is a MySQL peculiarity: latin1 does not mean ISO-8859-1:
mysql> SHOW CHARACTER SET like 'Latin1';
+---------+----------------------+-------------------+--------+
| Charset | Description | Default collation | Maxlen |
+---------+----------------------+-------------------+--------+
| latin1 | cp1252 West European | latin1_swedish_ci | 1 |
+---------+----------------------+-------------------+--------+
1 row in set (0.00 sec)
And Windows-1252 does not have a position for U+0099:
ISO-8859-1 (also called Latin-1) is identical to Windows-1252 (also
called CP1252) except for the code points 128-159 (0x80-0x9F).
ISO-8859-1 assigns several control codes in this range. Windows-1252
has several characters, punctuation, arithmetic and business symbols
assigned to these code points.
From West European Character Sets in the MySQL manual:
latin1 is the default character set. MySQL's latin1 is the same as the
Windows cp1252 character set. This means it is the same as the
official ISO 8859-1 or IANA (Internet Assigned Numbers Authority)
latin1, except that IANA latin1 treats the code points between 0x80
and 0x9f as “undefined,” whereas cp1252, and therefore MySQL's latin1,
assign characters for those positions. For example, 0x80 is the Euro
sign. For the “undefined” entries in cp1252, MySQL translates 0x81 to
Unicode 0x0081, 0x8d to 0x008d, 0x8f to 0x008f, 0x90 to 0x0090, and
0x9d to 0x009d.
In short: you cannot use a latin1 column to stored such character. Since you're already using UTF-8 in your app you should consider upgrading your database to utf8 or, even better, utf8mb4.

Answer 3

You can add support for UTF-8 character set during MySQL DB schema creation, as by default schema creation sometimes it doesn't use the UTF-8 char set.
CREATE DATABASE dbName
DEFAULT CHARACTER SET utf8
DEFAULT COLLATE utf8_general_ci;