I've been pulling my hear out over this problem for a few hours yesterday:
I've a database on MySQL 4.1.22 server with encoding set to "UTF-8 Unicode (utf8)" (as reported by phpMyAdmin). Tables in this database have default charset set to latin2. But, the web application (CMS Made Simple written in PHP) using it displays pages in utf8...
However screwed up this may be, it actually works. The web app displays characters correctly (mostly Czech and Polish are used).
I run: "mysqldump -u xxx -p -h yyy dbname > dump.sql". This gives me an SQL script which:
looks perfect in any editor (like Notepad+) when displaying in UTF-8 - all characters display properly
all tables in the script have default charset set to latin2
it has "/*!40101 SET NAMES latin2 */;" line at the beginning (among other settings)
Now, I want to export this database to another server running on MySQL 5.0.67, also with server encoding set to "UTF-8 Unicode (utf8)". I copied the whole CMS Made Simple installation over, copied the dump.sql script and ran "mysql -h ddd -u zzz -p dbname < dump.sql". After that, all the characters are scrambled when displaying CMSMS web pages.
I tried setting:
SET character_set_client = utf8;
SET character_set_connection = latin2;
And all combinations (just to be safe, even if it doesn't make any sense to me): latin2/utf8, latin2/latin2, utf8/utf8, etc. - doesn't help. All characters still scrambled, however sometimes in a different way :).
I also tried replacing all latin2 settings with utf8 in the script (set names and default charsets for tables). Nothing.
Are there any MySQL experts here who could explain in just a few words (I'm sure it's simple after all) how this whole encoding stuff really works? I read 9.1.4. Connection Character Sets and Collations but found nothing helpful there.
Thanks,
Matt
Did you try adding the --default-character-set=name option, like this:
mysql --default-character-set=utf8 -h ddd -u zzz -p dbname < dump.sql
I had that problem before and it worked after using that option.
Hope it helps!
Ugh... ok, seems I found a solution.
MySQL isn't the culprit here. I did a simple dump and load now, with no changes to the dump.sql script - meaning I left "set names latin2" and tables charsets as they were. Then I switched my original CMSMS installation over to the new database and... it worked correctly. So actually encoding in the database is ok, or at least it works fine with CMSMS installation I had at my old hosting provider (CMSMS apparently does funny things with characters encoding).
To make it work on my new hosting provider, I actually had to add this line to lib/adodb/drivers/adodb-mysql.inc.php in CMSMS installation:
mysql_query('set names latin2',$this->_connectionID);
This is a slightly modified solution from this post. You can find the exact line there as well. So it looks like mysql client configuration issue.
SOLUTION for me:
set this option in your php file, after mysql_connect (or after mysql_select_db)..
mysql_query("SET NAMES 'utf8'");
Related
I want to convert my database to utf-8. What I have done up to now is set the server to read utf-8 and the database is converted by using this query:
ALTER DATABASE database_name CHARACTER SET utf8 COLLATE utf8_unicode_ci;
Now all new information is seen and the things that were broken are now fine. The problem is that the old data is seen as �. This, in my opinion, is due to the fact that the old data is written in windows-1251 (I think at least and I am not 100% sure).
I found out that I need to dump the data:
mysqldump -uroot -p database -r utf8.dump
and then import it:
mysql -uroot -p --default-character-set=utf8 database
mysql> SET names 'utf8'
mysql> SOURCE utf8.dump
This is what I saw from here: https://makandracards.com/makandra/595-dumping-and-importing-from-to-mysql-in-an-utf-8-safe-way
The problem is that I have absolutely no idea where and how to do this.
All I have access is to the web hosting control panel and I have not set up anything on my computer. Therefore, I have no idea how to connect the database to the command shell and so on. What next steps should I do to convert the data to utf-8? Please, any detailed explanation would be great due to the fact that this is the first time for me doing something like this.
// I have a Mac and a Windows machine, but not a Linux at the moment.
Thank you!
The charset and collation of the database are the default for any subsequently created tables. The table setting are the defaults for columns.
For each table, so this:
ALTER TABLE table_name CONVERT TO utf8mb4;
I have searched online for a solution and none of the solutions that I found online seems to work here.
My problem is that I have a MySQL dump (SQL file) containing hebrew text values that is saved in UTF8 encoding, and when I import it using "source [file]" it saves the hebrew characters as question marks (???).
Now, when I look at the SQL file (cat [file]) I can see the hebrew characters properly.
Even when I try to copy & paste the SQL commands from the output that "cat" gave directly into the MySQL command line, it works as well.
It only fails when I use "SOURCE [file]" (which I need, because it is a HUGE file).
I have also tried the following:
mysql -uroot -p[pass] --default-character-set=utf8 , and then "SET NAMES utf8" and then "SOURCE [file]" - Gives question marks.
Login to mysql client, then do "SET NAMES utf8", "SET COLLATE utf8_bin" (this is the settings for all the tables in the DB) - Gives question marks.
CREATE DATABASE [db_name] DEFAULT CHARACTER SET UTF8 with the previous setting (section 2 above) - Gives question marks.
mysql -uroot -p[pass] --default-character-set=utf8 [db_name] < [file.sql] - Gives question marks.
set character_set_filesystem utf8 and then running source [file] - Gives question marks.
None of these works properly, the ONLY thing that works is if I do copy+paste directly from cat's output to mysql command line, which is not an option because of the length of the file (several hundreds of MB).
Please help, thanks!
Will be problem reading the file as < file.sql.
As documented about using MySQL in batch mode
If you are running mysql under Windows and have some special characters in the file < file.sql might cause problems, use this instead:
mysql -e "source file.sql" dbname ... --default-character-set=UTF8
Ok, I managed to solve this problem by creating a PHP "import" script.
First, I took the entire SQL file and split it into two files: structure (commands that create the tables and structure) and data.
Then, I just ran the entire structure.sql file using mysql_query, and took the data file, explode it by "\n" to get the seperate lines of all the INSERTs, and then ran them using a loop and mysql_query.
I didnt even need to include "SET NAMES utf8", figures out once I removed this line everything worked perfectly.
I know this isn't an ideal solution, but it is one that worked for me.
I need to transfer my mysql DB from windows server to Ubuntu server.
So i made export in phpmyadmin on win and imported *.sql file in linux.
In linux PMA all looks okay, tables are healthy, no errors, and charter set is (cp1251), russian data in tables looks how it needed.
But when i try to run select in php script there is only "???????" in result, and echo mysql_client_encoding() showing that charset is latin1.
Please, tell me where is latin1 can be seted?
Thanks for help.
UPD: I am using now mysql_set_charset('cp1251'); after each db connection, but its not an pefect solution. Maybe someone can offer other idea?
I've uploaded some data to a MySQL (5.5.15 for osx10.6) database using UTF8 encoding, though for some reason I had to specify its encoding as latin1 when I LOADed it.
I reckon this part is good because when I write to an OUTFILE, my unicode 'nu' characters come out OK in a terminal and in Vim.
However, when I look at them within a MySQL session, and when I try to edit the fields from Django admin, I get mangled characters (latin1?).
So, my question is: how to I tell a MySQL client and (especially) Django to read my database as UTF-8, the way it oughta?
At the command line, I tried
--default_character_set=utf8
and also
'SET NAMES UTF8;'
at the MySQL prompt, but they do not work.
When I look at VARIABLES LIKE 'char%', they're all set to utf8 apart from character_set_server which is latin1. If I set it to utf8.... that doesn't work either.
I'd be grateful if someone could give me some pointers here, especially about how to configure Django to talk to my database properly.
Thanks!
Add in your .cnf:
[mysqld]
character-set-server = utf8
collation-server = utf8_general_ci
skip-character-set-client-handshake
I'm trying to store unicode strings in a MySQL database (MySQL version 5.1.41-3ubuntu12.9 on an Ubuntu 10.04). This appears to work fine as long as I'm using the terminal to view the data. But, if I use MySQL Query Browser or Ruby on Rails to query the database all I get are garbage strings.
I've tried adding default-character-set = utf8 and character-set-server = utf8 to my my.cnf file and restarting MySQL, but that doesn't seem to help. My database.yml file has the line encoding: utf8 but I'm guessing this is not the issue considering the fact that I can't view the data properly in MySQL Query Browser either.
Any ideas on what to do?
I was having the exact same problem with no help from the internet. The data was showing the correct data in terminal window only but both MySQL Query Browser and PHP were showing strange characters.
Finally I realized that the data in the database was NOT saved using the correct character set. So I updated the code from where I was inserting and updating with the addition $link->set_charset('utf8'); where $link is the mysqli object.
Now the new data inserted is showing fine. I still don't understand why the MySQL Terminal was showing it correctly.
Anyway it works now!