I exported and then imported a wordpress mysql database from one server to another.
On the new server, a lot of the apostrophes have turned to question mark symbols ?. When I look at the data in the database itself, the apostrophes are normal like this ' so what would be causing those characters to look messed up?
Thanks
Perhaps run SHOW CREATE TABLE tablename on both servers. The difference might be related to the CHARSET.
You should use the corresponding collation in the select, for example
SELECT k
FROM t1
ORDER BY k COLLATE latin1_german2_ci;
One common collation name could be SQL_Latin1_General_Cp1254_CS_AS
here is a list of collation names:http://msdn.microsoft.com/en-us/library/ms180175.aspx
Related
I wish to save the IPA phonetic pronunciation against a word in a simple dictionary. I have a simple table where the IPA field is collated to UTF8_GENERAL_CI. I have a script that is running to extract the IPA from a source and write that into the table.
If I show the SQL prior to execution, the command looks fine. If I retrieve what IPA value after save, it is garbage. If I look at it using PHPMyAdmin, it is garbage. If I overwrite the value using PHPMyAdmin edit then it shows perfectly via PHPMyAdmin
UPDATE vocab_word SET phonetic='[ˈaːχənɐ]' WHERE id=4
What I see when retrieving it is '[ˈaËχənÉ]' and this is what I see within PHPMyAdmin.
So, I know I am getting it in the format I want, I know that if it is saved correctly, PHPMyAdmin can show it, but something is happening during the write to convert it.
I have tried :-
VARCHAR with UTF8_General_ci collation
VARCHAR with UTF8_bin
TEXT with UTF8_general_ci
TEXT with UTF8_bin
Within the SQL, I have tried simply using the value (as above) as well as using a UTF8_encode.
I am assuming there is a specific combination that I need to save the IPA text of encoding in the SQL and the database, and will continue to try combinations, however if someone can save me some time, I would really apprecaited it.
Regards
Chris H
It turned out to be a combination of things....
I needed to convert the character_set for the database to utf8mb4
I needed to convert the character_set for the table to utf8mb4
Collations needed to also be utf8mb4
plus the mysql connection needed to have it's character_set set.
Any one of these not done caused the characters to be mutated.
I have a huge database of book authors in which names of French authors have not been stored correctly and the French characters have been replaced by some strange characters!
Can I solve the problem by a SQL query? if yes, I do appreciate if you give me a clue.
Thanks,
Export your table data with a mysqldump
Change the character encoding of the dump file create table statement to utf8
drop the table or change the name to something like tablename_old (I recommend keeping the old table until after the experiment ;))
Import the modified dump file
Since french characters are all in UTF8 and you probably don't have a multi-byte encoding character set on your table, this should fix the issue.
You might be able to just run an alter table to change the encoding, but in my experience that can be a roll of the dice.
I am inserting some datas including turkish characters into a table and when I insert a data appearance of turkish characters confused me. For example for the character 'ğ', appearance on database is 'ð'. But when I get data and print it on the browser for testing this character shows up to be correct 'ğ' character. Do I have to worry about the situation ? (Btw I am using utf8mb4 as described in title)
If you are using phpmyadmin to see database entries, it is a common misconception.
You shouldn't worry and you can configure it from php.ini and phpmyadmin's settings. But as you said it is not much of a problem. I reccomend you to use toad or navicat for mysql for better results.
I have a database that contains data in different languages. Some languages use accents (like áéíóú) and I need to search in this data as the accents doesn't exist (search for 'campeon' should return 'campeón' as a valir result).
The problem is that the tables in my database (utf8_unicode_ci) are not storing utf8 characteres. If you see the data through phpmyadmin the words with accents looks like this: campeón
After some researching, I've found (in a StackOverflow question) that the problem is related to the inexistence of a SET NAMES [charset]. In fact, I've made some testings and if I set names to utf8, everything works as expected.
Well, I have the solution, what's the problem? The problem is that the database is in production, so there are thousands of strings in the database. If I change the character set the client will use, all already existing string will become invalid. The question is: is there any way to:
perform accent-insensitive searches in a database that uses a wrong charset like mine?
transform safely the data in the tables to the appropriate charset?
continue working with mixed charsets (latin1 and utf8) in the database, assuming that latin1 data will not be accent-insensitive?
If anybody has experience in any of the solutions I propose or has a new one, I'll be very thankful if share.
The problem being that the data was inserted using the wrong connection encoding, you can fix it by
Exporting the data using the wrong connection encoding, just like you have used it thus far, followed by
Importing the data using the correct utf8 connection encoding.
That will fix the encoding problem, after which search will work as expected.
What if you create a copy of the table at the beginning of your session, alter the copy's charset, perform all your queries from that, and then drop the table at the end of your session? I don't know how practical this would be - depends on how often you need to perform these queries and how big the table is.
How can I stop mysql from converting ' into ’ when I do an insert?
i believe it has something to do with charset or something?
I am using php to do the mysql_insert.
The single quotation mark you posted is called an 'acute accent', which is often converted from the generic single quotation mark by some web applications. It's a UTF8 character, which when inserted into a Latin-1 database translates to '’'. This means that you need to change MySQL's charset to UTF8, or alternatively change your website's charset to Latin-1. The former would be preferred:
ALTER DATABASE YourDatabase CHARACTER SET utf8;
ALTER TABLE YourTableOne CONVERT TO CHARACTER SET utf8;
ALTER TABLE YourTableTwo CONVERT TO CHARACTER SET utf8;
...
ALTER TABLE YourTableN CONVERT TO CHARACTER SET utf8;
Maybe someone will know the answer immediately, but I don't. However here are a few suggestions on what to examine (and possibly expand the question on)
When dealing with encodings and escaping you should include the full history of data
how was it created
what happened to it before the problem (did it have to go through backup, e-mail, was it created on a different server, OS, etc..; if it was transferred then was it as text file?)
The above is because anything that writes to a text file (browser, mysql client, web server, php application, to name a few layers that could have done it) can mess up character coding.
To troubleshoot, you can start eliminating, and thus the first step (in my book), is to
connect to mysql server using mysql command line client.
check the output of SHOW VARIABLES LIKE 'character_set%'
(so even in this simple environment you have 7 values that can influence how the data is parsed, stored and/or displayed
inspect SHOW CREATE TABLE TableName, and look for charset and collation info, both default for the table and explicit definition on columns
Having said all of the above, I don't think any western script would transcode a single quote character. So you might need to look at your escaping and other data processing.
EDIT
Most of the above from answer and discussion here
This is what I've done, and it worked for me:
First make sure that column containing ' is utf8_general_ci
Then add the mysql_set_charset to your code
$db=mysql_connect("localhost", $your_username, $your_password);
mysql_set_charset('utf8',$db);
mysql_select_db($your_db_name, $db);