i have a MYSQL DB. I am storing the multi lingual characters in a table. below is the sample code.
CREATE TABLE test_multi_lang
(
language_name varchar(500) CHARACTER SET utf8 COLLATE utf8_unicode_ci
);
SET NAMES 'utf8';
insert into test_multi_lang
(language_name )
values
('ตัวอย่าง');
insert into test_multi_lang
(language_name )
values
('नमूना');
SET NAMES 'utf8';
SET character_set_results = 'utf8',
character_set_client = 'utf8',
character_set_connection = 'utf8',
character_set_database = 'utf8',
character_set_server = 'utf8';
select * from test_multi_lang;
When i run the above code in TOAD for MYSQL 7.3.1.290 and select the result set it is displaying as ?????????. but when I run the same code through mysql console it is displaying the unicode characters properly. i changed the properties of my toad as
still i am not getting the result displayed in proper format.
how can i solve this problem ?
Thanks in advance.
After trying different options what worked for me is the following setting in my.ini configuration file.
character_set_server=utf8
I'm using xampp and my.ini is located under c:\xampp\mysql\bin folder
Here's the screenshot
To see your current settings run following command in Toad
SHOW VARIABLES LIKE '%CHARACTER_SET%'
after applying the changes it should look like this:
And this is how data looks after changing the setting.
Please note that you'll have to restart mySQL server as well as Toad for this setting to take effect. This change will not be applied retroactively to existing data. you'll have to recreate the old data.
SHOW CREATE TABLE
You will find that the table (or at least the text column) is not CHARACTER SET utf8. utf8 is required there.
This is a duplicate; read it for the cause and solution.
This is probably a display problem Toad is a non unicode software so we have to do a little setup.
You have to set NLS_LANG in regedit and add Arabic language from the region settings. For guidance you can follow the below steps:-
Windows -> Run -> regedit -> HKEY_LOCAL_MACHINE -> WOW6432Node -> ORACLE -> KEY_(YOUR HOME_ID) -> NS_LANG -> VALUE AMERICAN_AMERICA.AR8MSWIN1256
Control Panel -> Region -> Administrative -> change System Locale... -> select current system language (Arabic).
Restart Toad.
Enjoy & Thanks me later
Related
I have a problem with MySql.
My version of MYSql is : 5.7.33 - MySQL Community Server (GPL)
I have create a discord Bot in node.js, and i have a mistake when a new user with pseudo like this : legoshi🌌🌧
So i have try to follow this topic : How to fix "Incorrect string value" errors?
So i convert my Database in : utf8mb4_unicode_ci
And my error is still here.
At the begin my database was in utf8 and i have the error too.
code: 'ER_TRUNCATED_WRONG_VALUE_FOR_FIELD',
errno: 1366,
sqlMessage: "Incorrect string value: '\\xF0\\x9F\\x8C\\x8C\\xF0\\x9F...' for column 'user' at row 1",
sqlState: 'HY000',
index: 0,
sql: 'INSERT INTO registre (id, user, autohit, ultimate, platinium, `Date Inscription`) VALUES (210490816542670849, "legoshi🌌🌧", 0, 0, 0, CURRENT_TIMESTAMP())'
}
So i don't no how to change this. I have see a lot of topic and all seems to be fix with utf8mb4_unicode_ci but not in my case.
Thanks for you're help.
In MySQL, there are several places where you can set up a character set:
On the server level
On the database level
On the table level (for each table)
On the field level for all character-based fields
On your connection (telling the server what charset will be used in packets you send to the server)
Basically, server-level, database-level and table-level are just defaults for newly created items: New databases are generated with the server's default. New tables are created with the database's default, new fields are created with the table's default. However, only the field-level charset is what actually counts.
So first, you should make sure that the fields you want to store the data in actually are set up to utf8mb4_unicode_ci. Then, you need to connect to the server using exactly the same charset. Be aware that also the collation should match.
You can find out what character set is in use by issuing the following query:
SHOW VARIABLES LIKE 'character_set_%'
You'll see several variables indicating which default is set for various scopes. Have a look especially to the variables character_set_client and character_set_connection. If the connection does not have the correct character set specified, you need to set it up on connection.
It's a good practice to have all character sets match identically. Mixed values will sooner or later cause trouble.
To check the character set which is set up for the field, have it displayed with the command
SHOW CREATE TABLE registre
I don't think title could be more described better as tl;dr, because problem is a bit deeper.
I've got two databases (finnish language):
MSSQL (collation: SQL_Latin1_General_CP437_CI_AI)
MySQL (collation: utf_general_ci)
I've created BI project in vs2017, connected two databases and transfered tables from one to another, no problem. Except for 1 letter: "å" - instead it was "?". I cannot change any database collation so I am trying to find a way to transfer words with this letter.
What I've tried:
OLD DB Source -> ODBC Destination
Point "1" with "Data Conversion" block in between (with code page 1252)
Script Component, in which I have tried:
Insert with "_latin"
sql= "INSERT INTO db.words(Name) VALUES(_latin1'å')";
byte[] b = Encoding.UTF8.GetBytes(sql);
odbcCmd = new OdbcCommand(Encoding.UTF8.GetString(b), odbcConn);
odbcCmd.ExecuteNonQuery();
Insert without it
sql= "INSERT INTO db.words(Name) VALUES('å')";
byte[] b = Encoding.UTF8.GetBytes(sql);
odbcCmd = new OdbcCommand(Encoding.UTF8.GetString(b), odbcConn);
odbcCmd.ExecuteNonQuery();
Diferent ways of encoding
byte[] bytes = Encoding.GetEncoding(1252).GetBytes("å");
var myString = Encoding.GetEncoding(1252).GetString(bytes);
byte[] bytes2 = Encoding.Default.GetBytes("å");
var myString2 = Encoding.Default.GetString(bytes2);
Insert with COLLATE which got me error
insert into db.words(Name) values ("å" COLLATE latin1_swedish_ci) ;
and error:
System.Data.Odbc.OdbcException: „ERROR [HY000] [MySQL][ODBC 5.3(a) Driver][mysqld-5.7.21-log]COLLATION 'latin1_swedish_ci' is not valid for CHARACTER SET 'cp1250'”
Here is interesting part:
I can make insert with this letter in MySQL Workbench without a problem, and it will be inserted, but when I try to pass it from one database to another it is lost. I've set Data Viewers between Data Conversion and the letter was still there, and also when debugging script it was after encoding in string that were inserted to database.
Maybe someone got any idea what else I can try, because I feel like I have tried everything, and feel that the resolve of this problem is really close, but I just don't see it.
CP1250 does not include å; CP437 and utf8 do include it.
COLLATE is irrelevant -- it applies only to comparing and sorting.
Don't use any encode/conversion functions; instead, specify how the data is encoded.
I see 'code' -- but what is the encoding for the source in that language and/or editor?
Show us the hex of any strings in question.
Which direction are you trying to transfer?
What are the connection parameters for each database?
Ok, so I've spent the morning trying to change the default collation on my XAMPP setup.
Here's the problem: I'm using Format() in a view, to convert a double into a string
CREATE VIEW `test` AS
SELECT
Format(some_data_table.double_number,0) AS string_result
FROM some_data_table;
When I look at the returned column, its showing as utf8mb4_general_ci.
I've tried all manner of settings in my.ini and phpMyAdmin's config.inc.php
to no avail.
As a last resort, I'm prepared to add the collation parameter to view.
I'd be grateful for any tested solution
Ok - i'm going to post my own answer for anyone else who lands here:
(i had seen this somewhere else, but didn't trust it a t the time because there was no explanation).
When the SQL Format() turns a number into a string, it uses the variable character_set_results.
PMA's Variables Tab was showing this as "utf8" but then on a line below, it was saying (session value) = utf8mb4.
So i was aware that PMA was overriding the server default.
My real problem was that I could find no way to change this override - either by using the [mysqld] skip-character-set-client-handshake setting.. or by editing the php.config.inc file.
Today I had a breakthrough.. I established that if I used the same PMA to connect to and older MySQL server, the problem did not occur.
This suggested to be that PMA was forcing utf8mb4 on newer (capable) servers, but not older ones.
I did a text search of phpmyadmin for the string 'mb4' and found the following code in the class: phpMyAdmin/libraries/DatabaseInterface.class.php
// Skip charsets for Drizzle
if (!PMA_DRIZZLE) {
if (PMA_MYSQL_INT_VERSION > 50503) {
$default_charset = 'utf8mb4';
$default_collation = 'utf8mb4_general_ci';
} else {
$default_charset = 'utf8';
$default_collation = 'utf8_general_ci';
}
the PMA_MYSQL_INT_VERSION > 50503 seems to fit with my theory about older mysql versions, so i've backed up the file and edited the class replacing utf8mb4 with utf8 in this function.
phpMyAdmin is now showing what i want in its variables tab, and the Format() function is now returning what i expect.
(I won't give you a tested solution without a failing test case.)
Here's a possible explanation:
mysql> SELECT FORMAT(2e7, 0);
+----------------+
| FORMAT(2e7, 0) |
+----------------+
| 20,000,000 |
+----------------+
But you are working in a "locale" where the "thousands separator" is ., not ,.
The solution has nothing to do with COLLATION. Instead, look at the arguments to FORMAT().
mysql> SELECT FORMAT(2e7, 0, 'de_DE');
+-------------------------+
| FORMAT(2e7, 0, 'de_DE') |
+-------------------------+
| 20.000.000 |
+-------------------------+
I am guessing that MS Access and MySQL are assuming different "Locales", hence stumbling over the thousands separator, and possibly other differences.
References on Locale:
http://dev.mysql.com/doc/refman/5.7/en/locale-support.html
http://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_lc_time_names
I uploaded my website to the new server. It works perfectly on my test server at home, there is not a different setup the databases were copied over word for word. But on the site anything that is in Korean displays as ??????. The database stored it correctly and the pages all have <meta charset="UTF-8"> I can not figure out what I am missing.
EDIT: The text displays fine in the database when I use phpMyADMIN
In PDO (php api), you need set charset $conn->exec('SET CHARACTER SET utf8');.
PHP example:
<?php
//한국어/조선말
header('Content-Type: text/html; charset=utf8');
$username = 'user';
$password = 'password';
$host = 'domain';
$db = 'dbtest';
try {
$conn = new PDO('mysql:host=' . $host . ';dbname=' . $db . ';charset=utf-8', $username, $password);
$conn->exec('SET CHARACTER SET utf8');//This solve the problem
$stmte = $conn->prepare('SELECT id, text FROM test LIMIT 10');
$exec = $stmte->execute();
if ($exec) {
while($reg = $stmte->fetch(PDO::FETCH_OBJ)){
echo 'id: ' . $reg->id . '<br />';
echo 'text: ' . $reg->text . '<br /><hr />';
}
} else {
echo 'Error SELECT';
}
} catch(PDOException $e){
echo 'PDOException: ', $e->getMessage();
}
?>
Mysql example:
CREATE DATABASE `dbtest` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
USE `dbtest`;
CREATE TABLE IF NOT EXISTS `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`text` varchar(300) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `test` (`id`, `text`) VALUES (1, '한국어/조선말');
Use phpmyadmin in your server to verify that the DATABASE on your server is "utf8", see:
In your database must be something else.
if your database is correct (after checking with the utf8) then the problem is in some PHP file.
To resolve you should save all php files (both the major and the includes)
Save your html file (or php file) in "utf8 without boom", using notepad++, see:
Add in your PHP files (in top file):
<?php
header('Content-Type: text/html; charset=utf8');
?>
Files included should be saved in utf8-without-boom also, example:
<?php
include('YOUR FILE INCLUDED.php');// Save "YOUR FILE INCLUDED.php" in UTF8-without-boom
?>
Maybe its some page is in ANSI (for example "form.php").
Note: All PHP files must be in UTF8-without-boom format
Try adding lang attribute to your html tag
<html lang="ko">
The issue is most likely a different in database collation settings between your home test server & your new remote server. Meaning that while your database was transferred correctly, the way that data is then spit out of database is a whole other thing.
What is the data collation of the database giving you an issue? By default, most MySQL installs set latin1_swedish_ci instead of utf8_general_ci for newly created databases.
Change the collation of the database & try again.
ALTER DATABASE [name of your database] CHARACTER SET utf8;
If this is a specific table, the collation can be changed as so:
ALTER TABLE [name of your table] CONVERT TO CHARACTER SET utf8;
And if it is a specific column in a table:
ALTER TABLE [name of your table] MODIFY [name of your column] [other settings] CHARACTER SET utf8 COLLATE utf8_general_ci;
Or perhaps you could export the current database, create a new database with this command & reimport the data:
CREATE DATABASE [name of your database] CHARACTER SET utf8 COLLATE utf8_general_ci;
And if you want to make a permanent change to the MySQL install on the machine giving you an issue, go and edit my.cnf. The following would set the whole chain to UTF-8:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
EDIT: The original poster states that the connection & DB are all UTF8 clean. But what about trying an edit to the Apache default character set. Go here & open the character set file for Apache like so:
sudo nano /etc/apache2/conf.d/charset
And uncomment the line that looks like this:
#AddDefaultCharset UTF-8
So it looks like this:
AddDefaultCharset UTF-8
And restart Apache. This is not a great idea for a long term setup in my humble opinion, but if t solves the issue it indicates there is something in your codebase that can be changed to affect the same results without having to force Apache to force UTF8.
http://www.sqlfiddle.com/#!2/82f65/1
I tried this:
create table x(y varchar(100) character set utf8);
insert into x(y) values('爱');
But the chinese character doesn't appear:
select y from x;
Output:
Y
?
I'm the author of sqlfiddle.com. The problem was that I didn't have my connection string and default database encoding for mysql setup to properly handle UTF8. I have fixed this now, but because the fiddle you posted is still using the obsolete settings, you'll have to see it working here on my slightly-modified version of your fiddle:
http://www.sqlfiddle.com/#!2/e79e8/1
Your link might start working eventually, it just needs to clear out of the running memory and be reset. After no one hits it for a while it should be harvested and then ready to be built back up cleanly. Thanks!
FYI, the changes I had to make to get it to work were found here: http://www.compoundtheory.com/?action=displayPost&ID=421
The relavent bits where adding this to my connection string from java:
useUnicode=true&characterEncoding=UTF-8
And adding this to my create database statement:
create database my_new_database default CHARACTER SET = utf8 default COLLATE = utf8_general_ci;
It is working fine in mysql on my localhost. it may be due to mysql charset or some setting please check it.
If you have to run this query via program like php then
run query before select query
"SET NAMES utf8"
It will be return result properly
thanks
The Chinese character is not displaying in fiddle but in actual mysql database it is working fine. Kindly check your mysql version