Insert emoji character to MySQL DB from iOS app via Java - mysql

I'm working as a DBA on mysql 5.5.24 servers. Recently we faced one issue in production when one user tries to LogOn from iPhone and its phone name was having emoji character in itself.
Currently, in our system we have utf8 charset, which supports 3 bytes, but as its phone name was having emoji (4 bytes), the registration failed.
As mentioned in this link, we tried to reproduce same scenario in our development setup but still faces issues:
I tried after changing the charset of only that column in DB to utf8mb4 - No Success
I modified changing the charset of whole DB to utf8mb4 - No Success
Java Logs:
2013-01-08 11:21:54,547 ERROR [org.hibernate.util.JDBCExceptionReporter] (http-0.0.0.0) Incorrect string value: '\xF0\x9F\x98\x84' for column 'deviceDescription' at row 1
DB Logs:
insert into deviceDetails (deviceDescription, remoteStatus, deviceStatus) values ('?', 0, 1)
Note: Charset at Java side is still utf8. Can it be an issue?

See here. Also make sure that your connection character set is utfmb4.
Db characterset: utf8mb4
Client characterset: utf8mb4
Conn. characterset: utf8mb4

Here is the query to insert:
insert into homeDetails (homeId, homeDescription, routerMac, routerSSID, arpMac, homeDetailTS)values (123,'des','mactest',convert ('Emot😄RS'using utf8mb4),'aprtest', now());

Related

Bitnami database charset and collation issues ,site is created locally

I did made site on my localhost using Bitnami application,but when i export my database from PhpMyAdminm,and upload to hostting company they told me that they can’t import database because they get the error charset utf 8 collate utf8_general_ci,and i should change charset and collation in my bitnamidatabase.sql file in notepad,but when i opened i noticed that there is plenty lines of code that contains that so i give up,and i also noticed that in my phpmyadmin there is drop-down menu where i can change the charset ,in filed where are the info about database server is this Server charset: UTF-8 Unicode (utf8mb4) for the almost all files in that database collation is utf8mb4_unicode_520_ci,what should i do to fix that ,shoud i change type of collation ih the PhpMyAdmin or what?

Upgrade to MySQL 8. Unknown character set index for field '255' received from server. Exception

The mysql5 db has been used to latin1 and latin1_general_ci character encoding settings, all tables are used also this character set.
After migration to mysql8(create and restore dump from mysql5-to-mysql8) the error: Unknown character set index for field '255' received from server. occurs when doing connection to db.
Why does it happens?, I suppose that it may be related with a fact that mysql8 uses utf8mb4 character set encoding as a default.
But utf8mb4 encoding is more wide than latin1 so it must to be support to migration from latin1(1Byte) to utf8mb4(4Bytes) not vice verse.
The changing character set of the db to latin1 and collation to latin1_swedish_ci doesn't take affect.
There is some mysql8 params:
SHOW VARIABLES LIKE 'char%';
character_set_client utf8
character_set_connection utf8
character_set_database utf8mb4
character_set_filesystem binary
character_set_results utf8
character_set_server utf8mb4
character_set_system utf8
character_sets_dir C:\Program Files\MySQL\MySQL Server 8.0\share\charsets\
MySql8 connector driver is:
mysql-connector-java v. 3.1.14
I have this issue when I am executing scripts from my Java application. It is showing the same error as mentioned in the subject line.
The fix is(in my case,as we earlier use latin1): provide character encoding type in connection url.
?characterEncoding=latin1
Here is the list of available types which are used between java and mysql. This is the official documentation of java-mysql connector jar.
The only solution for this problem is
download the latest version of connector from the following website..
https://dev.mysql.com/downloads/connector/j/
When you go there there was a option to select operating system if you use windows then select the platform independent option then a download option appears ... download the zip file and extract it.Then go to your project and add this jar file into the libraries.So that's all your problem will be solved.
Hope this solution work for you...

Squeryl utf8mb4 support

I'm using Squeryl to work with a MySQL database. The tables are in utf8mb4 encoding. Now I want to insert some utf8 (4 byte) strings into the db through Squeryl. How do I do that?
I tried to set ?useUnicode=yes&characterEncoding=UTF-8 to my connection url but apparently, UTF-8 here is 3 byte to MySQL so it doesn't work.
I found this StackOverflow answer, but after some digging, I don't see anyway to append my queries with SET NAMES utf8mb4; (changing database config and environment is not an option)
Example string: อลิซร้องเพลงตามเลยค่ะ😂😂😂
Error when trying to insert the string:
Exception in thread "main" org.squeryl.SquerylSQLException: Exception while executing statement : Incorrect string value
Be sure not to connect as root.
Have this in my.cnf (in the [mysqld] section)
init_connect = SET NAMES utf8mb4

Arabic encoding with tomcat and mysql

i have tomcat server and mysql
when insert data with Arabic it appears like '???? ????'
i change MySQL database with
ALTER DATABASE database name CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
it did not work
and try in tomcat server.xml
change it to
Encouraging="UTF-8" on each Connector tag in the server.xml file
it still appears like '???? ??????'
can any one help me please

ERROR 1115 (42000): Unknown character set: 'utf8mb4'

I have a MySQL dump, which I tried to restore with:
mysql -u"username" -p"password" --host="127.0.0.1" mysql_db < mysql_db
However, this threw an error:
ERROR 1115 (42000) at line 3231: Unknown character set: 'utf8mb4'
This is lines 3231-3233:
/*!50003 SET character_set_client = utf8mb4 */ ;
/*!50003 SET character_set_results = utf8mb4 */ ;
/*!50003 SET collation_connection = utf8mb4_general_ci */ ;
I am using MySQL 5.1.69. How can I solve this error?
Your version does not support that character set, I believe it was 5.5.3 that introduced it. You should upgrade your mysql to the version you used to export this file.
The error is then quite clear: you set a certain character set in your code, but your mysql version does not support it, and therefore does not know about it.
According to https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html :
utf8mb4 is a superset of utf8
so maybe there is a chance you can just make it utf8, close your eyes and hope, but that would depend on your data, and I'd not recommend it.
You can try:
Open sql file by text editor find and replace all
utf8mb4 to utf8
Import again.
This can help:
mysqldump --compatible=mysql40 -u user -p DB > dumpfile.sql
PHPMyAdmin has the same MySQL compatibility mode in the 'expert' export options. Although that has on occasions done nothing.
If you don't have access via the command line or via PHPMyAdmin then editing the
/*!50003 SET character_set_client = utf8mb4 */ ;
bit to read 'utf8' only, is the way to go.
I am answering the question - as I didn't find any of them complete. As nowadays Unknown character set: 'utf8mb4' is quite prevalent as lot of deployments have MySQL less then 5.5.3 (version in which utf8mb4 was added).
The error clearly states that you don't have utf8mb4 supported on your stage db server.
Cause: probably locally you have MySQL version 5.5.3 or greater, and on stage/hosted VPS you have MySQL server version less then 5.5.3
The utf8mb4 character sets was added in MySQL 5.5.3.
utf8mb4 was added because of a bug in MySQL's utf8 character set.
MySQL's handling of the utf8 character set only allows a maximum of 3
bytes for a single codepoint, which isn't enough to represent the
entirety of Unicode (Maximum codepoint = 0x10FFFF). Because they
didn't want to potentially break any stuff that relied on this buggy
behaviour, utf8mb4 was added. Documentation here.
From SO answer:
Verification:
To verify you can check the current character set and collation for the DB you're importing the dump from - How do I see what character set a MySQL database / table / column is?
Solution 1: Simply upgrade your MySQL server to 5.5.3 (at-least) - for next time be conscious about the version you use locally, for stage, and for prod, all must have to be same. A suggestion - in present the default character set should be utf8mb4.
Solution 2 (not recommended): Convert the current character set to utf8, and then export the data - it'll load ok.
Just open your sql file with a text editor and search for 'utf8mb4' and replace with utf8.I hope it would work for you
maybe whole database + tables + fields should have the same charset??!
i.e.
CREATE TABLE `politicas` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`Nombre` varchar(250) CHARACTER SET utf8 NOT NULL,
-------------------------------------^here!!!!!!!!!!!
PRIMARY KEY (`ID`)
) ENGINE=MyISAM AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;
-------------------------------------------------^here!!!!!!!!!
As some suggested here, replacing utf8mb4 with utf8 will help you resolve the issue. IMHO, I used sed to find and replace them to avoid losing data. In addition, opening a large file into any graphical editor is potential pain. My MySQL data grows up 2 GB. The ultimate command is
sed 's/utf8mb4_unicode_520_ci/utf8_unicode_ci/g' original-mysql-data.sql > updated-mysql-data.sql
sed 's/utf8mb4/utf8/g' original-mysql-data.sql > updated-mysql-data.sql
Done!
Open your mysql file any edit tool
find
/*!40101 SET NAMES utf8mb4 */;
change
/*!40101 SET NAMES utf8 */;
Save and upload ur mysql.