I am using ckeditor 3.4 to insert data (text) to database and then display it on a page.
Problem: when I write (greek )in the ckeditor everything is fine. When I press the HTML button of the ckeditor again everything is fine (e.g. i see the actuall text typed not html entities). However when I save the data (and hence store them to the db) the stored data in the db are like this
"<p style="text-align: center;">
... σÏντομα πεÏισσότεÏες πληÏοφοÏίες...</p>
<p>
</p>"
Note: when I recall the data the are correctly displayed on the web page.
Actions taken so far:
1- the connection file to the db has the following: $conn->query("SET NAMES 'utf8'");
2- In the config.js of the ckeditor I have added the following lines
config.entities = false;
config.entities_greek = false;
config.entities_latin = false;
config.entities_processNumerical = false;
// Define changes to default configuration here. For example:
config.language = 'el';
// config.uiColor = '#AADC6E';
};
3- my webpages are set to: content="text/html;charset=utf-8"
4- db colation: utf8_unicode_ci / type MyIsam
I've been searching around but no luck.
I'd appreciate any help
Thank you all for your answers.
Solution was much simpler.
The right writing is SET NAMES UTF8 instead of SET NAMES 'utf8'
If you are using PHP or any other language that doesn't do this automatically, you need to invoke
SET NAMES 'UTF8'
on the connection before calling any statements, in order to use UTF-8 in your database.
Also make sure you are serving all pages as UTF-8 so that posted data is in UTF-8.
There are also some configuration parameters that controls how the data is sent and processed by the server, but I have never managed to get it to work without this statement.
se more here: http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html
EDIT: Ah, sorry, didn't see that you actually did this. If it is displayed correctly when you output it and your charset is set to UTF-8 on the page, then I'm assuming that you only view it in the DB with a tool that doesn't support UTF-8, or isn't configured for it? So what exactly is the problem right now?
Related
My MySQL database contains some Chinese symbols and such (non-ASCII symbols). When I view them in PHPMyAdmin, they look garbled. However, if I display them on my website with PHP using the regular mysqli API, it looks fine so I assume the data is uploaded/stored properly in the database, so maybe the server connection collation is incorrect.
My PHP code for opening the database connection is:
function openConnection(): mysqli
{
$databaseHost = "localhost";
$databaseUser = "root";
$databasePassword = '';
$databaseName = "my-database-name";
$connection = new mysqli($databaseHost, $databaseUser,
$databasePassword, $databaseName);
if ($connection->connect_error) {
die("Connection failed: " . $connection->connect_error);
}
return $connection;
}
My PHPMyAdmin server connection collation is the default utf8mb4_unicode_ci which seems to be reasonable as well. My tables are also created with the default utf8mb4_general_ci. Shouldn't that work fine for any input users might make?
Calling $connection->get_charset() in PHP also returns the correct charset:
If I export the database data in MyPHPAdmin, the export is also garbled in Notepad++, I made sure to view it with UTF-8 encoding. If I import the garbled export again, the database will show the data as garbled once more and on the website the data now also shows as garbled. In this case, an actually corrupted export happened.
How can I solve this encoding problem? Clearly PHP can handle UTF-8 properly, my Apache web server is also serving UTF-8 and my database is configured seemingly correctly as well but there is an issue with PHPMyAdmin or the database/database table collation.
It looks like the issue was entirely elsewhere since I'm supplying data to PHP with C++ code. The C++ code uses the nlohmann JSON libary to build the data submitted to the PHP script. The issue was my inability to specifically encode std::strings to UTF-8 like described here when putting data into a C++ JSON object. With that said, everything is now working as expected.
⚈ If using mysqli, do $mysqli_obj->set_charset('utf8mb4');
⚈ If using PDO do somethin like $db = new PDO('dblib:host=host;dbname=db;charset=utf8mb4', $user, $pwd);
⚈ Alternatively, execute SET NAMES utf8mb4
Any of these will say that the bytes in the client are UTF-8 encoded. Conversion, if necessary, will occur between the client and the database if the column definition is something other than utf8mb4.
More notes on PHP: http://mysql.rjweb.org/doc.php/charcoll#php
If you have specific garbling, see Trouble with UTF-8 characters; what I see is not what I stored
If you suspect the data being fed from PHP to Notepad, dump a few Chinese characters in hex and shown to us. I would expect every 4th character to be hex F0 or every 3rd to be between E3 and EA. (These are the first byte for 4-char and 3-char UTF-8 encoding of Chinese characters.)
Does Notepad properly handle UTF-8, or does it need a setting?
If you are in the "cmd" in Windows, you may need chcp 65001; see http://mysql.rjweb.org/doc.php/charcoll#entering_accents_in_cmd That way, more non-English characters will display correctly.
I have a DHTMLX grid on a page that saves data through a php connector file to a DB. The data from the grid is shown through xml encoding that is rendered in the PHP connector file.
Japanese words in the grid show up in Japanese but get saved as: ーダー
However they do stay in Japanese in the grid! (somehow...)
If I save something in the DB on php myadmin, it shows up in the grid as: ???
I checked and everything seems right...
DB fields: UTF-8 √
HTML headers: UTF-8 √
connector.php: UTF-8 √ (checked through network tab, devtools)
Is there anywhere else I should check?
When looking at the PHP file that gives me the DB values, I get XML data that's already garbled:
<rows><row id='00000000001'><cell><![CDATA[]]></cell><cell><![CDATA[??]]></cell><cell><![CDATA[33]]></cell><cell><![CDATA[]]></cell><cell><![CDATA[]]></cell><cell><![CDATA[?????????]]></cell>...
So maybe the problem lies before the data is received from the server. Does anyone know where I should look for the problem?
Were you expecting ーダー for ーダー? (Mojibake.)
Other times, do you get question marks?
Those two symptoms come from different causes. But both usually involve not declaring the client bytes to be utf8. In php, that can be done with mysqli_set_charset('utf8')
Question marks usually also involves failing to declare the column to be utf8.
To further diagnose, please do
SELECT col, HEX(col) FROM tbl WHERE ...
so we can see whether the text was mangled as it was inserted.
I'm using phpMyAdmin version 4.4.4 with MySQL 5.6 (charset is set to UTF-8 Unicode). The table in question has the collation set to utf8-general-ci and all fields are also set to utf8-general-ci collation as well. My php.ini file has default_charset = "UTF-8".
Despite all the UTF-8 settings for all three applications, unicode characters appear garbled when viewing a table within phpMyAdmin. So, instead of seeing ...
Søren
... in phpMyAdmin I see ...
Søren
Even though it displays garbled in phpMyAdmin, it displays correctly on the website. The only problem is with phpMyAdmin.
If I attempt to Insert a new record using phpMyAdmin and enter Søren in a text field, it displays like this within phpMyAdmin...
Søren
Which looks correct there, but, on the web page, it displays like this...
S�ren
The ø character is replaced with a question mark inside a black diamond instead of displaying the proper unicode character on the website.
What the heck is going on? How do I make phpMyAdmin display and insert the unicode characters properly into the table without mangling them? Thanks!
My php.ini file has default_charset = "UTF-8".
That only affects the charset used for some PHP built-in functions like htmlentities.
MySQL uses its own charset to decode stuff you send it. This can be set using $mysqli->set_charset('utf8') for mysqli, or mysql_set_charset('utf8') for the deprecated mysql module, or using charset=utf8 in the connection string in PDO.
I've setup a MySQL DB with utf8_unicode_ci collation, and all the tables and columns on it have de same collation.
My Doctrine config have SET NAMES utf8 as a connection option and my html files use utf8 charset.
The text saved on those tables contain accented characters (á,è,etc).
The problem is that when I save the content to the DB, it saves with strange characters, like when I try to save ISO in UTF8 table. (e.g.: NotÃcias)
The only workaround that i've found is to, utf8_decode before save, and utf8_encode before printing.
That means that, for some reason, something in between is messing up utf8 with iso.
What might be?
Thanks.
EDIT:
I've setup to encode before saving and decode before printing, and it prints correctly but in DB my chars change to:
XPTÓ -> XPTÓ
This makes searching in DB for "XPTÓ" impossible...
I would print bin2hex($string); at each step of the original workflow (i.e. without encode/decode steps).
Go through each of:
the raw $_POST data
the values you get after form validation
the values that get put in your bound Entity
the values you'd get from the db if you query it directly using PDO (get this from
$em->getConnection())
the values that get populated into your Entity on reload (can do this via $em->detach($entity); $entity = $em->find('Entity', $id);)
You'd be looking at the point at which the output changes, and focus your search there.
I would also double check:
On the db: SHOW CREATE TABLE 'table' shows CHARSET=utf8 for the whole table (and nothing different for the individual columns)
That the tool you use to see your database values (Navicat, phpMyAdmin) has got the correct encoding set.
I'm trying to move some Excel-Data to MySQL, but having troubles with encoding.
What I did:
Data export from OpenOffice 3.1 as csv (utf-8 encoded)
Import to phpMyAdmin via file upload (Table encoding: 'utf8_unicode_ci')
In phpMyAdmin's view mode, the data is displayed correctly (it is using utf-8 as charset):
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
When I try to display the Data on my webpage, I get a hash with a question mark in it.
System-Info
The language I try to get on my page:
German
MySQL client version: 5.0.32
My OS: MAC OS X 10.5.7
Server-Script: CakePHP v1.2.3.8166
Regards,
Benedikt
I had a similar symptom, my solution was to add
'encoding' => 'UTF8'
to config/database.php
I've had a similar problem to this. Most things are set correctly "out of the box", but I'd just like to point out the following things I found useful... I hit this problem while moving from dev to live:
You need to have matching Database encoding (2 items to check) as well as view encoding.
Your DB encoding is set up in the Schema.
Cake PHP can be manually forced to expect a type of encoding from the DB.
Cake PHP then needs to call the correct encoding for the view as well.
The DB settings are, of course, set up in your database. In my case, using MySQL workbench, this was simply a case of right clicking the schema and selecting "Alter Schema...". From there I could select the encoding/collation I wanted.
In Cake PHP the database encoding is set in app\Config\database.php. You should ensure that the DATABASE_CONFIG array (approx line 60), has the correct encoding enabled/selected. For example:
'encoding' => 'utf8'
Finally, the view needs to select the correct version of HTML to display. This is written into your templates from the file app\Config\core.php (approx line 82).
Configure::write('App.encoding', 'UTF-8');
Once all three parts are changed, you should have a consistent charset and hence display.
Hope that helps any people still searching for this.
Are you setting the connection to UTF-8 with mysql?
SET NAMES 'UTF-8'
$conn1 = mysqli_connect($this->db_host, $this->db_username, $this->db_password, $this->db_name);
Add this in connection
mysqli_set_charset($conn1,"utf8mb4");