Importing UTF8-encoded data into mysql is not working for me. UTF8 characters are corrupted. For example Nöthnagel is displayed as Nöthnagel
I have created a sql dump file to do the importing which contains UTF-8 encoded data. For example:
INSERT INTO `users` VALUES(1, 'Fred','Nöthnagel');
The sequence of bytes representing ö in the file is c3 b6 which I believe is correct, as it displays correctly in vim and in my bash shell which has these environment variables set:
$ env | grep -i utf
LANG=en_US.UTF-8
XTERM_LOCALE=en_US.UTF-8
The mysql db was created as follows:
mysql> CREATE DATABASE mydb CHARACTER SET utf8;
The mysql table was created so:
CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`first_name` varchar(30) NOT NULL,
`last_name` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `last_name` (`last_name`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;
I am importing the dump file like so:
mysql -u root -psecret mydb < mydump.sql
Please tell me what is missing from the above.
I think it might have something to do with collation as well, but I'm not sure. In my case it certainly did, since I had to support cyrillic.
Try this, worked for me:
Set initial collation while creating the target database to utf8_unicode_ci
Add SET NAMES 'utf8' COLLATE 'utf8_unicode_ci'; to the top of your sql file
Run mysql -u root -p --default-character-set=utf8 yourDB < yourSQLfile.sql
One more thing, in order to properly get the UTF-8 data form your database, you'll have to modify your connection string as well. For example:
mysql.url=jdbc:mysql://localhost:3306/nbs?useJvmCharsetConverters=false&useDynamicCharsetInfo=false&useUnicode=true&characterEncoding=UTF-8&characterSetResults=UTF-8&useEncoding=true
Additionally, take a look at what my problem was.
Use this command for import utf8 table to database :
mysql -u USERNAME -pPASSWORD --default_character_set utf8 DATABASE < file.sql
The problem was solved by adding this at the top of the sql file:
SET NAMES utf8;
I had a similar problem. There are a number of variables that should be UTF8, not only the database, those include the client, the connection, ther server ...etc.
The solution to your problem is described in this article. The described solution is portable, so it does not only work for utf8, but for all other character sets. You may need to modify it to fit your needs.
Related
I'm migrating a small Gyroscope app from MySQL to MariaDB. I asked this question earlier on why MariaDB would not start on latin1 as server encoding (MariaDB won't start when character-set-server = latin1 is in my.cnf), but now I've come to terms with using just utf8.
After importing the data dump:
mysql -uwebuser -p ezine < dump.sql
, what's supposed to be autotööstus shows up as autot??stus. In addition I got this error:
illegal mix of collations (latin1_general_ci implicit) and (utf8_general_ci coercible), storing ??
Attempt 1: I removed all the occurrences of charset=latin1 in the data dump, and the import terminated early with this error:
Specified key was too long; max key length is 767 bytes
Attempt 2: added back charset, but set it to charset=utf8, according to this post: MySQL Convert latin1 data to UTF8
The import works, but autotööstus is now showing as autotööstus
How do I interpret the data dump as latin1 encoded content but deposit into a utf8 storage?
If you cannot use latin1 as the default system encoding for whatever reason, here's a quick solution:
Leave the data dump as-is. Even though it's latin1 encoded, it can be imported as utf8.
mysql> create database ezine character set utf8 collate utf8_general_ci;
mysql -uwebuser -p --default-character-set=utf8 ezine < dump.sql
Now in your web app, you should correctly see autotööstus
From now on, the "native currency" of your database is utf8. When dumping the database, you have to be careful, make sure the exported file is still latin1 encoded.
Otherwise you'll end up with the over-encoded data, such as autotööstus.
mysqldump -uwebuser -p --default-char-set=latin1 ezine > dump.sql
I am a beginner when it comes to databases, so please bear with me. I'm trying to set up a database and import some tables from a file tables.sql. Some of the Columns in tables.sql have Swedish letters in them (Ä, Ö) and the problem is that I get the following:
Ä = ä
Ö = ö
First I begin to check the character set of the server:
mysql> show variables like 'character_set_server';
The server is configured to character set 'Latin-1'. I must mention that I have no control over the server more than to create a database. So I guess I have to create my database and specify the character set of the database.
This is how I proceed:
mysql> create database db;
mysql> alter database db character set utf8 collate utf8_swedish_ci;
I double checked that my tables.sql have charset utf-8 by executing:
file -bi allsok_tables.sql
And then I load it into the database by:
$ mysql -u [username] -h [hostname] -P [port] -p db < tables.sql
when I create my tables in tables.sql I use engine = InnoDB (don't know if this is relevant or not). However if I now select everything from the table TableTest
mysql> select * from TableTest
I get these weird characters instead of the Swedish characters. I appreciate any help right now.
Thanks in advance!
UPDATE:
If I insert a value manually into a table it works e.g.
mysql> insert into TableTest values ('åäö');
So the problem seems to be with the .sql-file. Right?
$ mysql ... --default-character-set=utf8 < tables.sql
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
MySQL needs to know what encoding the data you're sending it is in. To do this, you need to set the connection encoding. When connecting to MySQL from a client, you usually run a SET NAMES query or use an equivalent call on your API of choice to do so. On the command line, the --default-character-set option does this. It needs to be set to whatever encoding your file is in.
The title saiz it all, right? :)
Details:
I'm looking for a SQL query I can use in phpmyadmin. Need to Change all my fields in one database from latin1_swedish_ci to utf8_general_ci
The main problem in your case is when you change the collation/character set from the table the data inside the table are latin1 and the table is utf8 until then.
What you can try is to make a dump of the table with the old character set.
mysqldump -uuser -p --default-character-set=latin1 dbname > dump.sql
and then import the database with the new character set like this:
mysql -uuser -p --default-character-set=utf8 dbname_test < dump.sql
To test i would import the dump to a test database. If the characters are not correct after the import. Use an Editor like pspad or notepad++ and change the file encoding to UTF-8. After this you can try to import another time.
The most times encodings are very annoying but i hope you can solve it.
When you only have phpmyadmin use the "Export" function and make the same with phpmyadmin. Export in the actual encoding and try to import in the new encoding but then you have to change the file encoding i think.
Change the default collation to utf8_general_ci in MySql:
Open the my.ini file. (C:\xampp\mysql\bin\my.ini)
Find the text [mysqld] and add the below lines.
[mysqld]
character-set-server = utf8
collation-server = utf8_general_ci
The above two lines will select a character set and collation at server startup. These settings apply server-wide and apply as the defaults for databases created by any application, and for tables created in those databases.
when I create a new table, it always is 'latin-swedish-ci' by default. my goal is that this table can store utf-8 text. I tried this way, give me error .
cd C:\mysql-5.5.14-winx64\bin
mysqld --standalone --console --default-storage-engine=InnoDB --default-character-set=utf8
[ERROR] mysqld: unknown variable 'default-character-set=utf8'
Your Database probably has the default charset on latin1 and default collation set to latin1-swedish-ci and so altering the table should help
http://dev.mysql.com/doc/refman/5.1/en/alter-database.html
ALTER DATABASE `MyDatabaseName`
CHARACTER SET utf8
COLLATE utf8_general_ci;
You can always specify column encodings/collations within CREATE TABLE statement. This has additional benefit of making your data structure more portable in case you ever need to move it to another server.
I need to change the default charset of MySQL. I want the new database and all tables to use UTF-8. When I create django unit tests, the tests fail while creating the DB because it is set to latin1.
I can't do that in my.cnf. I use Ubuntu and MySQL installed from repository.
create database `something` default charset=utf8;
http://dev.mysql.com/doc/refman/5.0/en/create-database.html
Try:
# Alter your database
ALTER DATABASE database_name DEFAULT CHARSET=utf8;
SHOW CREATE DATABASE database_name;
CREATE DATABASE `zzz_test` /*!40100 DEFAULT CHARACTER SET utf8 */