I am using media temple and I create my tables like this using a PHP file (encoded in UTF-8 without BOM):
CREATE TABLE table_name (
...
) ENGINE=InnoDB CHARACTER SET utf8 COLLATE utf8_unicode_ci
I have two situations:
1 - Inserted some rows into table via php code. Turkish characters are displayed weirdly in phpMyAdmin, however when I print them on browser, they look correct.
2 - I add some data with Turkish characters into the table via phpMyAdmin SQL Query tab. This time I see correct characters in phpMyAdmin, however, when I print table rows to browser, I got quesionmarks instead of Turkish characters.
My browser uses UTF-8 as character encoding. I tried "utf8_turkish_ci" as collation for the table but no effect. I changed phpMyAdmin language to Turkish but it didn't work neither. When I export database from Media Temple, all Turkish character are replaced with weird ones. Do I missing something?
I solve my problem by using mysqli. I was in a paradox while using mysql extension in PHP such that when I get correct characters in phpMyAdmin, I get wrong on browser or vice versa. I just convert codes, nothing more (connect, set names to utf8 and do whatever you want). Every operation looks fine now. I don't why but changing extension solved the problem.
Related
I'm using the Encoding UTF-8 Unicode (utf8mb4) and Collation utf8mb4_unicode_520_ci for both tables and fields in my MySQL database.
When I export the database from Sequel Pro and open the exported .sql file in a text editor my test character ๐ appears correctly, but when I import the file back into Sequel Pro it appears as ???? both in Sequel Pro and in my PHP/MySQL app.
In the import window I've tried Autodetect and Unicode (UTF-8) without success. Any ideas?
Also, is there any newer encoding out there that I should use instead () and is there any benefit of using utf8mb4_unicode_520_ci instead of just utf8mb4_unicode_ci?
Edit / Here's a picture of what I'm trying to do. It seems like my "odd" character is on track all the way until I'm trying to import the .sql file back into Sequel PRO.
The COLLATION does not matter except for ordering. The CHARACTER SET does matter, since this is a 4-byte code.
Somehow CHARACTER SET utf8 got involved, in spite of what you say. See "question marks" in Trouble with utf8 characters; what I see is not what I stored for the likely causes.
Do SELECT HEX(...) ... to verify that that character was actually stored as hex F09D8C86.
Provide SHOW CREATE TABLE so we can verify that the column is utf8mb4.
And, let's see the connection parameters.
I wish to save the IPA phonetic pronunciation against a word in a simple dictionary. I have a simple table where the IPA field is collated to UTF8_GENERAL_CI. I have a script that is running to extract the IPA from a source and write that into the table.
If I show the SQL prior to execution, the command looks fine. If I retrieve what IPA value after save, it is garbage. If I look at it using PHPMyAdmin, it is garbage. If I overwrite the value using PHPMyAdmin edit then it shows perfectly via PHPMyAdmin
UPDATE vocab_word SET phonetic='[หaหฯษnษ]' WHERE id=4
What I see when retrieving it is '[รหaรรโกรโขnร]' and this is what I see within PHPMyAdmin.
So, I know I am getting it in the format I want, I know that if it is saved correctly, PHPMyAdmin can show it, but something is happening during the write to convert it.
I have tried :-
VARCHAR with UTF8_General_ci collation
VARCHAR with UTF8_bin
TEXT with UTF8_general_ci
TEXT with UTF8_bin
Within the SQL, I have tried simply using the value (as above) as well as using a UTF8_encode.
I am assuming there is a specific combination that I need to save the IPA text of encoding in the SQL and the database, and will continue to try combinations, however if someone can save me some time, I would really apprecaited it.
Regards
Chris H
It turned out to be a combination of things....
I needed to convert the character_set for the database to utf8mb4
I needed to convert the character_set for the table to utf8mb4
Collations needed to also be utf8mb4
plus the mysql connection needed to have it's character_set set.
Any one of these not done caused the characters to be mutated.
So, I have a database and I use Navicat. We have a simple PHP website which is a few years old and we've upgraded the site to UTF8.
We have 'activities' on the site which handle UTF8 special characters perfectly, but we also have 'comments' on the site and curly single quotes and other special characters show me a ๏ฟฝ.
The database was converted to UTF via:
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
When I look at both databases in Navicat, I can see both are UTF8 and utf8_general_ci.
When I design the table I can see the 'activities' table I can see the cell is a mediumText and is setup with UTF8. When I design the 'comments' section, the cell that isn't working is a Blob and it doesn't have any character encoding info.
We're doing a pretty basic SELECT and then displaying via $vairable[column].
Does anyone know why the 'activities' would work perfectly with UTF8 and the 'comments' would have issues? We're not doing anything super fancy to either of them.
I have tried converting the Blob to a text field, but when I do that the database then escapes it'self when it's outputting to the page, so as soon as there is a single quote in the text it cuts off.
I have tried things like utf8_encode, stripslashes, mysql_real_escape_string, htmlentities, htmlspecialchars, but I'm not sure any of them would help anyway.
Thanks!
blob means binary large object. Binary data does not have any encoding in raw.
So you have latin1 or whatever data in a blob, and you show it and treat it like utf-8 data.
You need to manually convert the data using PHP or whatever.
Here is a good article from the performanceblog that describes what you can do:
http://www.mysqlperformanceblog.com/2013/10/16/utf8-data-on-latin1-tables-converting-to-utf8-without-downtime-or-double-encoding/
If you have problems firing your queries, use the console instead of phpMyAdmin and don't forget the connection encoding through SET NAMES
master> ALTER TABLE t CONVERT TO CHARACTER SET utf8, CHANGE comment comment TEXT;
master> SET NAMES utf8;
I have all my database/tables and columns set to UTF-8_general_ci collation set.
Conditions that I Faced :-
When I insert hindi data manually by phpmyadmin, I can see the the hindi characters in phpmyadmin, while question marks when seen on webpage generated by PHP
In the same table when I insert data by HTML/PHP Forms I see some unrecognizable words in english something like cc2faa;(something like this) and Correct Hindi on Webpage.
For the large data we have a script that reads from txt files and insert the data in the table in this , I see characters like ร ยคลร ยคยพร ยคยจร ยคยพ in phpmyadmin but Hindi On webpage.
Now the main problem is :-
Data has gone under changes online by forms and now I need this data to export to excel and give to the client but I am getting ร ยคลร ยคยพร ยค in excel instead of Hindi Characters.
Note :-
All English characters are working fine and as it is everywhere.
My CHARACTER SET is utf8 For all tables.
I tried to change the collation to UTF-8_bin but that too doesn't helped me in anyway.
Encoding on the browser is UTF-8, and I have already sent the headers for UTF-8 encoding.
I have seen many posts about utf8 problem but no one seem to have this weird different behavior problem.
Please Do I have any rescue from this?? Or finally have to give the PHP reports of the data??
Please help!!
When I insert hindi data manually by phpmyadmin, I can see the the hindi characters in phpmyadmin, while question marks when seen on webpage generated by PHP
PHP probably generates the question marks because the encoding of the database connection is not utf-8. How to fix this depends on the database library you use; if you use MySQLi use mysqli_set_charset('utf8'), if PDO you add charset=utf8 to the DSN...
In the same table when I insert data by HTML/PHP Forms I see some unrecognizable words in english something like cc2faa;(something like this) and Correct Hindi on Webpage.
For the large data we have a script that reads from txt files and insert the data in the table in this , I see characters like ร ยคลร ยคยพร ยคยจร ยคยพ in phpmyadmin but Hindi On webpage.
These are likely caused by the same problem as above: the PHP forms and the script connect to the database using the default encoding, probably latin1. Then they insert utf-8 encoded text, but since MySQL thinks you are using latin1, it encodes the text into utf-8 again, and inserts this doubly encoded text into the table.
So: PHP sends "เคเคพเคจเคพ" to MySQL telling it's latin1, and MySQL goes and converts it to utf-8, resulting in "ร ยคลร ยคยพร ยคยจร ยคยพ". Later PHP asks MySQL return the value, and since the connection is again using latin1, MySQL takes "ร ยคลร ยคยพร ยคยจร ยคยพ" and decodes it to latin1. Then PHP pretends that this latin1 string is actually utf-8 and displays "เคเคพเคจเคพ".
Again, the solution is setting the encoding of the connection to utf-8. And this depends on what you use to access the database.
If you need to export your data as Excel file, use the PHP class php-export-data by Eli Dickinson, http://github.com/elidickinson/php-export-data. It is pretty nifty and so far I had have no problems exporting weird character sets with it.
I am inserting some datas including turkish characters into a table and when I insert a data appearance of turkish characters confused me. For example for the character 'ฤ', appearance on database is 'รฐ'. But when I get data and print it on the browser for testing this character shows up to be correct 'ฤ' character. Do I have to worry about the situation ? (Btw I am using utf8mb4 as described in title)
If you are using phpmyadmin to see database entries, it is a common misconception.
You shouldn't worry and you can configure it from php.ini and phpmyadmin's settings. But as you said it is not much of a problem. I reccomend you to use toad or navicat for mysql for better results.