MySQL Collation Issue - mysql

Question

In my company, the tables in the database were poorly created. Each table has a different collation and charset.
This is very bad, sure, but it makes queries loose a lot of performance til the point the server crashes (and it isn't even a great database...).
I would like to know if there are any good MySQL tools, commands or procedures for converting table collation and charset.
Just executing the alter table and executing convert is braking special characters. Is it normal or I am doing something wrong?
EDIT:
As exemple: I have a table finance with uft8 collation and a table expense with latin swedish. Each table has between 1000 and 5000 rows. The following query takes about 15 second to execute:
select ex.* from expense ex
inner join finance fin on fin.ex_id = ex.id
Executing much complexer queries with bigger tables runs much faster when they have the same collation.
EDIT 2:
Another error in the database: row ids are all varchar(15), not int.

Answer 1

I know the fun of inheriting legacy schemas created by folks who think 'collation' is some form of illness.
The best option is to export the table with it's data to a SQL dump file using good ole' mysqldump. Then modify the create statements manually in the dump file to set the character set and collation. I'm a big fan of 'utf8'. If the dump file is huge, use command line stuff like sed to efficiently edit the file without having to open it in an editor.
Then drop the existing table re-import the modified dump.
Any other way you do this in my experience can be a roll of the dice.
This might be a good time to convert them all to the same storage engine as well or upgrade your MySQL server to 5.5.

Answer 2

I don't recommend to use a "tool" to fix this.
BEFORE YOU DO ANYTHING DUMP YOUR DB TO HAVE A BACKUP IN CASE YOU MESS IT UP ;)
You can streamline your character sets and collation two ways
Method 1: Move your data
Create a completely new database with correct character sets and collations configured in all tables
Fill your new tables with INSERT SELECT statements
e.g.
INSERT INTO newdatabase.table SELECT * FROM olddatabase.table
MySQL will automatically convert your data into the correct character set
Method 2: Alter your tables
If you change the character set of a existing table, all existing contents will be converted as well
e.g.
old table
CREATE TABLE `myWrongCharsetTable` (
`name` varchar(255) COLLATE latin1_german1_ci NOT NULL DEFAULT ''
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_german1_ci;
put some data in for demo
INSERT INTO `myWrongCharsetTable` (`name`) VALUES ( 'I am a latino string' );
INSERT INTO `myWrongCharsetTable` (`name`) VALUES ( 'Mein Name ist Müller' );
INSERT INTO `myWrongCharsetTable` (`name`) VALUES ( 'Mein Name ist Möller' );
SELECT * FROM myWrongCharsetTable INTO outfile '/tmp/mylatinotable.csv';
On a UTF-8 console I do this
# cat /tmp/mylatinotable.csv
I am a latino string
Mein Name ist M▒ller
Mein Name ist M▒ller
right, strange charset.. this is latin 1 displayed on a utf-8 console
# cat /tmp/mylatinotable.csv | iconv -f latin1 -t utf-8
I am a latino string
Mein Name ist Müller
Mein Name ist Möller
Yep, all good
So how do I fix this now??
ALTER TABLE myWrongCharsetTable
MODIFY name varchar(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
DEFAULT CHARSET = utf8 COLLATE utf8_unicode_ci;
That's it :)
Writing the outfile again
mysql> SELECT * FROM myWrongCharsetTable INTO outfile '/tmp/latinoutf8.csv';
Query OK, 3 rows affected (0.01 sec)
mysql> exit
Bye
dbmaster-001 ~ # cat /tmp/latinoutf8.csv
I am a latino string
Mein Name ist Müller
Mein Name ist Möller
Worked, all fine and we're happy
EDIT:
There's actually another method
Method 3: Dump, modify and reload your data
If you're good with sed and awk you can automate this, or edit the file manually
# dump the structure, possibly routines and triggers
mysqldump -h yourhost -p -u youruser --no-data --triggers --skip-comments --routines yourdatabase > database_structure_routines.sql
# dump the data
mysqldump -h yourhost -p -u youruser --no-create-info --skip-triggers --skip-routines yourdatabase > database_data.sql
Now open the database_structure_routines.sql in an editor of your choice and modify the tables to your needs
I recommend to drop all the comments like /*!40101 SET character_set_client = utf8 */ in your dumpfile because this could overwrite table defaults
When you're done, create a new database and structure
mysql > CREATE DATABASE `newDatabase` DEFAULT CHARSET utf8 COLLATE utf8_unicode_ci;
mysql > use `newDatabase`
mysql > ./database_structure_routines.sql;
Don't forget to recheck your tables
mysql > SHOW CREATE TABLE `table`;
If that's all right you can reimport your data, charset conversion again will be done automatically
mysql -h yourhost -p -u youruser newDatabase < database_data.sql
Hope this helps

Answer 3

You could try using CONVERT or CAST to change the charset - create a new column and use CAST to fill the new column with new corrected charset.
http://dev.mysql.com/doc/refman/5.0/en/charset-convert.html