I'm using mysqldump to dump my database that contains UTF8MB4 columns with UTF8MB4 data. When I import this .sql file into a new database with UTF8MB4 support, all UTF8MB4 characters are converted into ????. Anybody got a clue about how to make MySQL and import work with UTF8MB4?
You should specify the character set with --default-character-set=utf8mb4 option when using mysqldump.
$ mysqldump --default-character-set=utf8mb4 -uusername -p database > dump.sql
Since MySQL 8.0 default charset for mysqldump is utf8mb4, so the problem should not appear anymore.
However in MySQL 5.7 default charset for mysqldump is utf8, so there you should explicitly change it as in Henridv answer (--default-character-set=utf8mb4).
Related
I'm migrating a small Gyroscope app from MySQL to MariaDB. I asked this question earlier on why MariaDB would not start on latin1 as server encoding (MariaDB won't start when character-set-server = latin1 is in my.cnf), but now I've come to terms with using just utf8.
After importing the data dump:
mysql -uwebuser -p ezine < dump.sql
, what's supposed to be autotööstus shows up as autot??stus. In addition I got this error:
illegal mix of collations (latin1_general_ci implicit) and (utf8_general_ci coercible), storing ??
Attempt 1: I removed all the occurrences of charset=latin1 in the data dump, and the import terminated early with this error:
Specified key was too long; max key length is 767 bytes
Attempt 2: added back charset, but set it to charset=utf8, according to this post: MySQL Convert latin1 data to UTF8
The import works, but autotööstus is now showing as autotööstus
How do I interpret the data dump as latin1 encoded content but deposit into a utf8 storage?
If you cannot use latin1 as the default system encoding for whatever reason, here's a quick solution:
Leave the data dump as-is. Even though it's latin1 encoded, it can be imported as utf8.
mysql> create database ezine character set utf8 collate utf8_general_ci;
mysql -uwebuser -p --default-character-set=utf8 ezine < dump.sql
Now in your web app, you should correctly see autotööstus
From now on, the "native currency" of your database is utf8. When dumping the database, you have to be careful, make sure the exported file is still latin1 encoded.
Otherwise you'll end up with the over-encoded data, such as autotööstus.
mysqldump -uwebuser -p --default-char-set=latin1 ezine > dump.sql
I need to convert a MyISAM latin 1 DB into a MyISAM utf8 db
following another question I exported my DB without any encoding specification
mysqldump -uroot -p --skip-opt --set-charset --skip-set-charset mydb > mydb_utf8.sql
changed my DB to default charset utf8
now I need to specify on import that i want MyISAM tables ( default setting is InnoDB )
mysql -uroot -p --default-character-set=utf8 --default-storage-engine=MyISAM mydb < mydb_utf8.sql
--default-storage-engine=MyISAM give me this error "mysql: unknown variable 'default-storage-engine=MyISAM'"
--default-storage-engine looks like its a mysqld startup option not a mysql start option..
Open your SQL file with a text editor and on the top
SET default_storage_engine=MyISAM;
I've got some problem in one of the TYPO3 Polish sites with character encoding. There was no setDBinit=SET NAMES utf8; parameter set in configuration.
Everything works okay (frontend & backend) but the export from the database. All international characters are corrupted when I search database via PhpMyAdmin or try to export database with data.
The official page http://wiki.typo3.org/UTF-8_support#SET_NAMES_utf8.3B says:
Without SET NAMES utf8; your TYPO3 UTF-8 setup might work, but chances are that database content entered after the conversion to UTF-8 has each international character stored as two separate, garbled latin1 chars.
If you check your database using phpMyAdmin and find umlauts in new content being shown as two garbled characters, this is the case. If this happens to you, you cannot just add the above statement any more. Your output for the new content will be broken. Instead you have to correct the newly added special chars first. This is done most easily by just deleting the content, setting the option as described above and re-entering it.
Is there any other way to repair corrupted characters? There is a lot of content to edit now...
I tried almost every combination of export encoding and converting to another encoding and so on and so far I failed.
You can try mysqldump to convert from ISO-8859-1 to utf-8:
mysqldump --user=username --password=password --default-character-set=latin1 --skip-set-charset dbname > dump.sql
chgrep latin1 utf8 dump.sql (or when you prefer sed -i "" 's/latin1/utf8/g' dump.sql)
mysql --user=username --password=password --execute="DROP DATABASE dbname; CREATE DATABASE dbname CHARACTER SET utf8 COLLATE utf8_general_ci;"
mysql --user=username --password=password --default-character-set=utf8 dbname < dump.sql
At shell command prompt:
mysqladmin -u"username" -p"password" --default-character-set=utf8 CREATE my_db_schema
--default-character-set=utf8 seems to have no effect and I don't understand why.
Database gets created, but character set is latin1 with collation latin1_swedish_ci.
I found this question, which would seem to be the same issue, but even when I tried a non-root user as the selected answer suggested, I get identical behavior:
MySQL connection character set problems
(I'm using Windows and MariaDB if that makes any difference)
I have tried these mysqladmin.exe clients:
MariaDB 5.3.2 for Win32 (ia32) with default character set latin1 (no .ini)
MySQL 5.0.77 for linux-gnu (i686) with default character set utf8
In both cases, --default-character-set=utf8 or --default-character-set=latin1 do NOT override the MySQL server's .ini/.cnf settings.
As a workaround I'd suggest running:
echo "CREATE DATABASE my_db_schema DEFAULT CHARACTER SET utf8" | mysql -uusername -ppassword
--default-character-set=utf8 seems to have no effect and I don't understand why.
Database gets created, but character set is latin1 with collation latin1_swedish_ci.
This options does not influence the character of a datatabase, table or column when they are created.
The default-character-set is the character set of the connection to the server -- it ensures values you select from the database come through to the client with the correct encoding for display.
On the surface I'd say this appears to be a mysqladmin bug. I would let the MariaDB devs know about it.
http://kb.askmonty.org/en/reporting-bugs has general instructions about reporting bugs (ignore the bit about using the mysqlbug script, since it is not available on Windows).
P.S. And if the bug exists in MariaDB it likely also exists in MySQL.
The title saiz it all, right? :)
Details:
I'm looking for a SQL query I can use in phpmyadmin. Need to Change all my fields in one database from latin1_swedish_ci to utf8_general_ci
The main problem in your case is when you change the collation/character set from the table the data inside the table are latin1 and the table is utf8 until then.
What you can try is to make a dump of the table with the old character set.
mysqldump -uuser -p --default-character-set=latin1 dbname > dump.sql
and then import the database with the new character set like this:
mysql -uuser -p --default-character-set=utf8 dbname_test < dump.sql
To test i would import the dump to a test database. If the characters are not correct after the import. Use an Editor like pspad or notepad++ and change the file encoding to UTF-8. After this you can try to import another time.
The most times encodings are very annoying but i hope you can solve it.
When you only have phpmyadmin use the "Export" function and make the same with phpmyadmin. Export in the actual encoding and try to import in the new encoding but then you have to change the file encoding i think.
Change the default collation to utf8_general_ci in MySql:
Open the my.ini file. (C:\xampp\mysql\bin\my.ini)
Find the text [mysqld] and add the below lines.
[mysqld]
character-set-server = utf8
collation-server = utf8_general_ci
The above two lines will select a character set and collation at server startup. These settings apply server-wide and apply as the defaults for databases created by any application, and for tables created in those databases.