I uploaded my website to the new server. It works perfectly on my test server at home, there is not a different setup the databases were copied over word for word. But on the site anything that is in Korean displays as ??????. The database stored it correctly and the pages all have <meta charset="UTF-8"> I can not figure out what I am missing.
EDIT: The text displays fine in the database when I use phpMyADMIN
In PDO (php api), you need set charset $conn->exec('SET CHARACTER SET utf8');.
PHP example:
<?php
//한국어/조선말
header('Content-Type: text/html; charset=utf8');
$username = 'user';
$password = 'password';
$host = 'domain';
$db = 'dbtest';
try {
$conn = new PDO('mysql:host=' . $host . ';dbname=' . $db . ';charset=utf-8', $username, $password);
$conn->exec('SET CHARACTER SET utf8');//This solve the problem
$stmte = $conn->prepare('SELECT id, text FROM test LIMIT 10');
$exec = $stmte->execute();
if ($exec) {
while($reg = $stmte->fetch(PDO::FETCH_OBJ)){
echo 'id: ' . $reg->id . '<br />';
echo 'text: ' . $reg->text . '<br /><hr />';
}
} else {
echo 'Error SELECT';
}
} catch(PDOException $e){
echo 'PDOException: ', $e->getMessage();
}
?>
Mysql example:
CREATE DATABASE `dbtest` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
USE `dbtest`;
CREATE TABLE IF NOT EXISTS `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`text` varchar(300) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `test` (`id`, `text`) VALUES (1, '한국어/조선말');
Use phpmyadmin in your server to verify that the DATABASE on your server is "utf8", see:
In your database must be something else.
if your database is correct (after checking with the utf8) then the problem is in some PHP file.
To resolve you should save all php files (both the major and the includes)
Save your html file (or php file) in "utf8 without boom", using notepad++, see:
Add in your PHP files (in top file):
<?php
header('Content-Type: text/html; charset=utf8');
?>
Files included should be saved in utf8-without-boom also, example:
<?php
include('YOUR FILE INCLUDED.php');// Save "YOUR FILE INCLUDED.php" in UTF8-without-boom
?>
Maybe its some page is in ANSI (for example "form.php").
Note: All PHP files must be in UTF8-without-boom format
Try adding lang attribute to your html tag
<html lang="ko">
The issue is most likely a different in database collation settings between your home test server & your new remote server. Meaning that while your database was transferred correctly, the way that data is then spit out of database is a whole other thing.
What is the data collation of the database giving you an issue? By default, most MySQL installs set latin1_swedish_ci instead of utf8_general_ci for newly created databases.
Change the collation of the database & try again.
ALTER DATABASE [name of your database] CHARACTER SET utf8;
If this is a specific table, the collation can be changed as so:
ALTER TABLE [name of your table] CONVERT TO CHARACTER SET utf8;
And if it is a specific column in a table:
ALTER TABLE [name of your table] MODIFY [name of your column] [other settings] CHARACTER SET utf8 COLLATE utf8_general_ci;
Or perhaps you could export the current database, create a new database with this command & reimport the data:
CREATE DATABASE [name of your database] CHARACTER SET utf8 COLLATE utf8_general_ci;
And if you want to make a permanent change to the MySQL install on the machine giving you an issue, go and edit my.cnf. The following would set the whole chain to UTF-8:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
EDIT: The original poster states that the connection & DB are all UTF8 clean. But what about trying an edit to the Apache default character set. Go here & open the character set file for Apache like so:
sudo nano /etc/apache2/conf.d/charset
And uncomment the line that looks like this:
#AddDefaultCharset UTF-8
So it looks like this:
AddDefaultCharset UTF-8
And restart Apache. This is not a great idea for a long term setup in my humble opinion, but if t solves the issue it indicates there is something in your codebase that can be changed to affect the same results without having to force Apache to force UTF8.
Related
I have a MySql database dump saved as a text file and am trying to find a way of extracting the indivdual pdf documents stored in one of the tables. All my research online so far has drawn a blank.
The data in the exported text file is in the following format:
DROP TABLE IF EXISTS `codocs`;
CREATE TABLE `codocs` (
`ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
`COCODE` varchar(8) NOT NULL,
`FILENAME` varchar(100) NOT NULL,
`DATE` date NOT NULL,
`USER` varchar(10) NOT NULL,
`DOCUMENT` mediumblob NOT NULL,
PRIMARY KEY (`ID`),
KEY `oc` (`COCODE`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
LOCK TABLES `codocs` WRITE;
/*!40000 ALTER TABLE `codocs` DISABLE KEYS */;
INSERT INTO `codocs` (`ID`, `COCODE`, `FILENAME`, `DATE`, `USER`, `DOCUMENT`)
VALUES
(1,’123456’,’document-2016-01-18.pdf','2016-01-21’,’user1’,X’8CB7638C2840B32D3AB66DDBB66DDBB66DDBB6BDC7B66DDBB6B1C7336F9F736EDECD4DBE1FE7477752D555ABBB562A59D5A40A2262B48C74CC0450A48747734B04508C040C04F64656 …
D2495CC3D8C1FCB8845D1D6F6C717E5EFB493B431B1250782FFFC12FD518D0E4EBF951D3B98F3C7971C1235F54B793172A427FF0F'),
(2,’234567’,’document-2016-01-18.pdf','2016-01-22’,’user1’,X’8CF763702E4EF02D0AC7B6ED64C7B66DDB7E62DBB6EDECD8C98E6DDBB66D3B797FE79C5BEFAD5BF5FF70AA66BAAA7B7AD674AD999A5A4DAE282A4EC744CF4204437E7038BB4804C344C448646F6C4504C3CB4B04C3A0EAE900206210317231B2B137FFCF57343207381331FF9 …
971C1235F54B793172A427FF0F'),
(3,’…
Any assistance would be greatly appreciated.
Update: 20220112
I have since restored the database from the sql dump and have subsequently created the following php files to try to display the pdfs stored in the codocs table:
db.php - contains the mysql database connection - this is working
records_list.php - lists all the records in the codocs table including a button on each returned row to view the stored pdf - this is working
view_pdf.php - receives the ID for the record clicked on from the records_list.php file and passes the selected record ID to the SELECT statement and displays the correct (presumably, as different data is returned for each separate record clicked on in the records_list.php file) raw mediumblob code stored in the database -
this is not working as intended
The following code is for the view_pdf.php file:
<?php
$pdf_id = $_REQUEST['pdfID'];
require_once "db.php";
if(isset($pdf_id)) {
$myID = $pdf_id;
$sql = "select * from codocs where ID='" . $myID . "'";
if (!$result=mysqli_query($con, $sql)){
echo mysqli_error($con);
} else {
$row = mysqli_fetch_array($result);
echo $row["DOCUMENT"];
mysqli_close($con);
}
}
?>
As mentioned just the raw mediumblob data appears to be being returned.
If the following line is replaced:
echo $row["DOCUMENT"];
with
echo '<object data="data:application/pdf;base64,'.base64_decode($row['DOCUMENT']).'" type="application/pdf" style="height:1000px;width:100%"></object>';
or
echo base64_decode($row['DOCUMENT']);
it makes no difference. Raw code continues to be returned.
If the original line of code referred to above is replaced with
header('Content-type: application/pdf');
echo $row["DOCUMENT"];
a downloadable pdf is offered and can be saved but is unreadable with the following warning: "This PDF document might not be displayed correctly." and the following error: "Unable to open document file:///...document.pdf. File type unknown (application/octet-stream) is not supported."
Can anyone advise how the code above can be amended to allow the retrieval of the stored pdf files?
Is the X that precedes the single quotations marks shown surrounding the mediumblob data in the sql dump file of any significance?
Any assistance would be greatly appreciated.
Further Update 20220112:
The following are example unreadable pdf restores but generate 'pdf' files of differing sizes:
Record 554:
Using the following replacement code:
header('Content-type: application/pdf');
echo $row["DOCUMENT"];
generates an unreadable file 82.2Kb in size.
Using the following replacement code:
header('Content-type: application/pdf');
echo '<object data="data:application/pdf;base64,'.base64_decode($row['DOCUMENT']).'" type="application/pdf" style="height:1000px;width:100%"></object>';
generates an unreadable file 15.6Kb in size.
Using the following replacement code:
header('Content-type: application/pdf');
echo '<object data="data:application/pdf;base64,'.base64_encode($row['DOCUMENT']).'" type="application/pdf" style="height:1000px;width:100%"></object>';
generates an unreadable file 109.7Kb in size.
Any thoughts on helping to resolve the issue would be very welcome.
I've searched for about the last 3 hours for a solution, but it doesn't work.
MySQL doesn't support utf8mb4 (this is one solution I can't test).
Thank you!
Here is an example DB connection using PHP and MySQL 5.5.3:
public function_construct($host, $db, $user, $pass){
try {
$this->conn = new PDO("mysql:host = {$host};
dbname = {$db};
charset = utf8mb4",
$user,
$pass);
} else { exit(); }
}
As of release 5.5.3 utf8mb4 is fully backwards compatible with utf8. If you are working with an existing database look for the MySQL configuration file and change instances of 'utf8' to 'utf8mb4' accordingly.
Use Mysql blob field, save images directly there, although I personally don't like saving images in DB, save links to files instead.
$img = mysql_escape_string(file_get_contents('imagefile.gif'));
Then you would insert this $img into db.
i have a MYSQL DB. I am storing the multi lingual characters in a table. below is the sample code.
CREATE TABLE test_multi_lang
(
language_name varchar(500) CHARACTER SET utf8 COLLATE utf8_unicode_ci
);
SET NAMES 'utf8';
insert into test_multi_lang
(language_name )
values
('ตัวอย่าง');
insert into test_multi_lang
(language_name )
values
('नमूना');
SET NAMES 'utf8';
SET character_set_results = 'utf8',
character_set_client = 'utf8',
character_set_connection = 'utf8',
character_set_database = 'utf8',
character_set_server = 'utf8';
select * from test_multi_lang;
When i run the above code in TOAD for MYSQL 7.3.1.290 and select the result set it is displaying as ?????????. but when I run the same code through mysql console it is displaying the unicode characters properly. i changed the properties of my toad as
still i am not getting the result displayed in proper format.
how can i solve this problem ?
Thanks in advance.
After trying different options what worked for me is the following setting in my.ini configuration file.
character_set_server=utf8
I'm using xampp and my.ini is located under c:\xampp\mysql\bin folder
Here's the screenshot
To see your current settings run following command in Toad
SHOW VARIABLES LIKE '%CHARACTER_SET%'
after applying the changes it should look like this:
And this is how data looks after changing the setting.
Please note that you'll have to restart mySQL server as well as Toad for this setting to take effect. This change will not be applied retroactively to existing data. you'll have to recreate the old data.
SHOW CREATE TABLE
You will find that the table (or at least the text column) is not CHARACTER SET utf8. utf8 is required there.
This is a duplicate; read it for the cause and solution.
This is probably a display problem Toad is a non unicode software so we have to do a little setup.
You have to set NLS_LANG in regedit and add Arabic language from the region settings. For guidance you can follow the below steps:-
Windows -> Run -> regedit -> HKEY_LOCAL_MACHINE -> WOW6432Node -> ORACLE -> KEY_(YOUR HOME_ID) -> NS_LANG -> VALUE AMERICAN_AMERICA.AR8MSWIN1256
Control Panel -> Region -> Administrative -> change System Locale... -> select current system language (Arabic).
Restart Toad.
Enjoy & Thanks me later
I have moved my server from dreamhost to Godaddy recently
When I was with Dreamhost it was no problem when my visitors submitting a form in my website using other language than English.
For example they use Thai language when they submit the form I receive it with no problem.
but what happen now when they submit any form I will receive a text like ?????????? so I though it might be just the setting on the php.ini file if that was set correctly and I see it was using UTF-8
; PHP's built-in default is text/html
default_mimetype = "text/html"
default_charset = "UTF-8"
and
exif.encode_unicode = "UTF-8"
exif.decode_unicode_motorola = "UCS-2BE"
exif.decode_unicode_intel = "UCS-2LE"
;exif.encode_jis =
exif.decode_jis_motorola = JIS
exif.decode_jis_intel = JIS
but I can't figure out why this is happen while all was set correctly
and each of my html/php header file I have included
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
Any suggestion what I missed?
To put some more information that
If I use the FTP client such as Filezilla there will be another problem that when I edit the file .php or .html the content in the file if it has any other language than English it will go all like question mark ??????? but if I edit it from the web like Net2ftp then it has no problem on that
I had a similar problem with PHP + MySQL, MySQL refused to properly store UTF8 strings. The solution was to tell MySQL each time after connection, to use UTF8. I believe the SQL command to do so was SET CHARACTER SET UTF8, or SET NAMES UTF8 but I'm not sure, it was a long ago.
Edit: Found some old PHP code of mine, I hope it is helpful:
function sql_connect(){
global $host,$user,$pass,$dbname,$conn;
// connect to mysql
$conn=mysql_connect($host,$user,$pass);
// if cannot connect, exit
if(!$conn){
die('Cannot connect to MySQL server: '.mysql_errno().' - '.mysql_error());
};
// if cannot set to utf8, exit
if(!mysql_query('set names utf8;',$conn)){
mysql_close($conn);
die('Cannot set to UTF8: '.mysql_errno().' - '.mysql_error());
};
// if cannot select database, exit
if(!mysql_select_db($dbname,$conn)){
mysql_close($conn);
die('Cannot select database: '.mysql_errno().' - '.mysql_error());
};
};
function sql_to_html($x){
return(htmlspecialchars(stripslashes($x),ENT_QUOTES,"UTF-8"));
};
Don't forget to set PHP's internal encoding to UTF8 using iconv_set_encoding. Your php.ini preferences only affect the HTTP headers that are sent with your document! Finally, make sure you pass the encoding to mangling functions like htmlentities.
Can someone explain me when I set everything to UTF-8 I keep getting those damn ���
MySQL
Server version: 5.1.44
MySQL charset: UTF-8 Unicode (utf8)
I create a new database
name: utf8test
collation: utf8_general_ci
MySQL connection collation: utf8_general_ci
My SQL looks like this:
SET SQL_MODE="NO_AUTO_VALUE_ON_ZERO";
CREATE TABLE IF NOT EXISTS `test_table` (
`test_id` int(11) NOT NULL,
`test_text` text NOT NULL,
PRIMARY KEY (`test_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
INSERT INTO `test_table` (`test_id`, `test_text`) VALUES
(1, 'hééélo'),
(2, 'wööörld');
My PHP / HTML:
<?php
$db_conn = mysql_connect("localhost", "root", "") or die("Can't connect to db");
mysql_select_db("utf8test", $db_conn) or die("Can't select db");
// $result = mysql_query("set names 'utf8'"); // this works... why??
$query = "SELECT * FROM test_table";
$result = mysql_query($query);
$output = "";
while($row = mysql_fetch_assoc($result)) {
$output .= "id: " . $row['test_id'] . " - text: " . $row['test_text'] . "<br />";
}
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html lang="it" xmlns="http://www.w3.org/1999/xhtml" xml:lang="it">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>UTF-8 test</title>
</head>
<body>
<?php echo $output; ?>
</body>
</html>
Try to set charachter encoding after mysql_connect function like this:
mysql_query ("set character_set_client='utf8'");
mysql_query ("set character_set_results='utf8'");
mysql_query ("set collation_connection='utf8_general_ci'");
I set everything to UTF-8
Not quite.
You have to tell mysql your client's encoding.
As a matter of fact, you don't have to set up "everything" in utf-8. You can have your tables in latin1 and output in utf-8. Or contrary.
Very flexible.
But you have to set up client's encoding explicitly.
So, that's why it works with set names utf8. Because this query setting up client's encoding. And let Mysql know that data must be sent in utf-8. Pretty sensible, huh?
Also I have to mention your SQL dump. It needs same setting. Just SET NAMES somewhere at the top. Because you are sending these queries from some client too. And this client's encoding needs to be set up as well.
And one more thing to mention: be sure your server sending proper encoding in the Content-type header. You didn't set it to UTF-8 too.
I didn't see a "SET NAMES 'utf8';" query just after connecting to your database.
Try it, may work for you.
I would say that you forget to set the content type encoding of your PHP file to utf-8:
header('Content-Type: text/html; charset=utf-8');
Or is the encoding error within the MySQL database?
If only loading the data returns the wrong results, you can use the queries mentioned before or this line of code to enable UTF-8 for queries:
$mysqli->set_charset('utf8');
I hope that is what you needed.
take a look at the mysql_set_charset function. Perhaps you need to call it before you retreive the data.
You want to check the current charset using mysql_client_encoding and when needed mysql_set_charset. Or just never mind the checking and blindly go with setting.