While creating database for my website I used below syntax.
CREATE DATABASE myDatabase DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Now while my client is entering arabic characters, he see some weird output. I am using JSF 2.0 for web pages.
What changes do I need to make so that I can enter Arabic or any characters in my site and it get stored in DB.
Edit 1
While I am printing the data, I am seeing output as شسÙ?بشسÙ? بشسÙ?ب شسÙ?ب
Note:
I am using web application using JSF 2.0
You should set UTF8 charset for the connection before the inserting/reading data -
SET NAMES utf8;
INSERT INTO table VALUES(...);
SELECT * FROM table;
Use N'' when you insert data values, This denotes that the subsequent string is in Unicode (the N actually stands for National language character set)
INSERT INTO table VALUES(N'ArabicField');
I think you must use cp1256_general_ci instead of utf8_general_ci,
and don't forget to set the collation of the database and all fields that may contain Arabic words to utf8_general_ci.
Related
I currently do have an address table in MYSQL, with its Character Set set to 'utf8' and Collation to 'utf8_unicode_ci'. There exists a column name Address and I am trying to store the city name Łódź into the Address column. I tried to key in directly into the table at SQLyog Community 64, as well as using the tool MYSQL for Excel but it keeps showing the error 'Incorrect string value'.
I have tried to set the Character Set set to 'utf8mb4' and Collation to 'utf8mb4_unicode_ci'and it still gives me the same error.
Any help on how should I set the character set and collation in order to store Łódź? This city name is just one of many examples, and moving forward I may experience other similar characters as well. What can I use for a universal character set?
(utf8 and utf8mb4 work equally for Polish characters.)
You have not provided enough details about the flow of the characters, but the following should provide debugging for MySQL:
Trouble with utf8 characters; what I see is not what I stored
When stored correctly, the utf8 (or utf8mb4) encoding for Łódź is hex C581 C3B3 64 C5BA.
I am unable to find the exact solution for MySQL
The thing is the column supports by default UTF-8 encoding which consists of 3 bytes. The Indian Rupee Symbol, since it is new has a 4 byte encoding. So we have to change the character encoding to utf8_general_ci by,
ALTER TABLE test_tb MODIFY COLUMN col VARCHAR(255)
CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL;
After executing the above query simply execute the following query to insert the symbol,
insert into test_tb values("₹");
Ta-Da!!!
You are talking Oracle, yet it is tagged MySQL. Which do you want? And what language and/or client tool are you using?
Copy and paste it. Which Rupee do you like? ৲ ৳ ૱ ௹ ₨ ꠸
Probably you want this one:
UNHEX('E282A8') = '₨'
which is U+20A8 or 8360 in non-MySQL contexts
You need to have CHARACTER SET utf8 on the table/column.
You need to have done SET NAMES utf8 (or equivalent) when connecting.
Simplest way to do it is, utf8mb4 stores all the symbols
ALTER TABLE AsinBuyBox CONVERT TO CHARACTER SET utf8mb4;
I've been using for a long time a database/connection with the wrong encoding, resulting the hebrew language characters in the database to display as unknown-language characters, as the example shows below:
I want to re-import/change the database with the inserted-wrong-encoded characters to the right encoded characters, so the hebrew characters will be displayed as hebrew characters and not as unknown parse like *"× ×תה מסכי×,×× ×©×™× ×ž×¦×™×¢×™× ×œ×™ כמה ×”× "*
For the record, when I display this unknown characters sql data with php - it shows as hebrew. when I'm trying to access it from the phpMyAdmin Panel - it shows as jibrish (these unknown characters).
Is there any way to fix it although there is some data already inserted in the database?
That feels like "double-encoded" Hebrew strings.
This partially recovers the text:
UNHEX(HEX(CONVERT('× ×תה מסכי×,××' USING latin1)))
--> '� �תה מסכי�,��
I do not know what leads to the � symbols.
Please do SELECT col, HEX(col) FROM ... WHERE ...; for some cell. I would expect שלום to give hex D7A9D79CD795D79D if it were correctly stored. For "double encoding", I would expect C397C2A9C397C593C397E280A2C397C29D.
Please provide the output from that SELECT, then I will work on how to recover the data.
Edit
Here's what I think happened.
The client had characters encoded as utf8; and
SET NAMES latin1 lied by claiming that the client had latin1 encoding; and
The column in the table declared CHARACTER SET utf8.
Yod did not jump out as a letter, so it took a while to see it. CONVERT(BINARY(CONVERT('×™×™123' USING latin1)) USING utf8) -->יי123
So, I am thinking that that expression will clean up the text. But be cautious; try it on a few rows before 'fixing' the entire table.
UPDATE table SET col = CONVERT(BINARY(CONVERT(col USING latin1)) USING utf8) WHERE ...;
If that does not work, here are 4 fixes for double-encoding that may or may not be equivalent. (Note: BINARY(xx) is probably the same as CONVERT(xx USING binary).)
I am not sure that you can do anything about the data that has already been stored in the database. However, you can import hebrew data properly by making sure you have the correct character set and collation.
the db collation has to be utf8_general_ci
the collation of the table with hebrew has to be utf8_general_ci
for example:
CREATE DATABASE col CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE TABLE `col`.`hebrew` (
`id` INT NOT NULL AUTO_INCREMENT,
`heb` VARCHAR(45) NOT NULL,
PRIMARY KEY (`id`)
) CHARACTER SET utf8
COLLATE utf8_general_ci;
INSERT INTO hebrew(heb) values ('שלום');
Would anyone know of a reliable method (with mySQL or otherwise) to select rows in a database that contain Japanese characters? I have a lot of rows in my database, some of which only have alphanumeric characters, some of which have Japanese characters.
Rules when you have problem with character sets:
While creating database use utf8 encoding:
CREATE DATABASE _test DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Make sure all text fields (varchar and text) are using UTF-8:
CREATE TABLE _test.test (
id INT NOT NULL AUTO_INCREMENT,
name VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE = MyISAM;
When you make a connection do this before you query/update the database:
SET NAMES utf8;
With phpMyAdmin - Choose UTF-8 when you login.
set web page encoding to utf-8 to make sure all post/get data will be in UTF-8 (or you'll have to since converting is painful..). PHP code (first line in the php file or at least before any output):
header('Content-Type: text/html; charset=UTF-8');
Make sure all your queries are written in UTF8 encoding. If using PHP:
6.1. If PHP supports code in UTF-8 - just write your files in UTF-8.
6.2. If php is compiled without UTF-8 support - convert your strings to UTF-8 like this:
$str = mb_convert_encoding($str, 'UTF-8', '<put your file encoding here');
$query = 'SELECT * FROM test WHERE name = "' . $str . '"';
That should make it work.
Following on to the helpful answer NickSoft, i had to set the encoding on the db connection to get it to work.
&characterEncoding=UTF8
Then the SET NAMES utf8; seemed to be redundant
As teneff stated, just use SELECT.
When installing MySQL, use UTF-8 as charset. Then, choosing utf8_general_ci as collation should do the work.
As Frosty stated, just use SELECT.
Look up the lowest and highest valued Japanese characters in the Unicode charts at http://www.unicode.org/roadmaps/bmp/ and use REGEXP. It may use several different regions of characters to get the whole Japanese character set. As long as you use the UTF-8 charset and utf8_general_ci collation, you should be able to use a REGEXP '[a-gk-nt-z]' where a-g represents one range of Unicode characters from the charts, k-n represents another range, etc.
There is limited number of japanese characters. You can search for these using
SELECT ... LIKE '%カ%'
Alternatively you can try their hexadecimal denomination -
SELECT ...LIKE CONCAT('%',CHAR(0x30ab),'%')
You may find useful this UTF-8 Japanese subset
http://www.utf8-chartable.de/unicode-utf8-table.pl?start=12448
Supposing you're using UTF-8 character set for fields, queries, results...
How do I store Unicode in free edition of MySQL?
There doesn't seem to be nvarchar type as in SQL Server. Is Unicode not supported in MySQL? I tried using text but that too is not working.
You need to choose a utf8_* character set for your table. Text and memo fields will then automatically be stored in UTF-8. Support for UTF-16 is coming in mySQL 6.
The character set for a given string column (CHAR, VARCHAR or *TEXT) is determined by its character set and/or collation. This is a quite intense topic so it's best to read the documentation I linked. For example:
CREATE TABLE t1
(
col1 CHAR(10) CHARACTER SET utf8 COLLATE utf8_unicode_ci
)
will create a table t1 with a col1 that stores its content in UTF-8 encoding.
Have you tried setting names after connection? What was the outcome of tryng to store unicode characters? Connect to mysql server and type this:
SET NAMES UTF8;
This should turn on "support" for utf8. Then try storing utf data.
MySQL supports UTF-8.
Use varchar and set encoding for the column (it's quite easy in phpMyAdmin)
http://www.oreillynet.com/onlamp/blog/2006/01/turning_mysql_data_in_latin1_t.html - I think this could help you
Use this Query
alter table `DBNAME`.`TblName` Engine=InnoDB checksum=1 comment='' delay_key_write=1 row_format=dynamic charset=utf8 collate=utf8_unicode_ci
I follow my own style of coding, so please take that into account when following my lead.
First I created the MySql database.
after creating database, in mysql command prompt, give the command:
SET NAMES = UTF8;
table and columns to store unicode are to be set with collation property as utf8-utf_general_ci. Each and every column which meant to store unicode, is to be selected and set collation property to utf8-utf_general_ci.
Now in C#.
my connection string is set as usual but with addition of a single attribute like this:
constring = #"SERVER=localhost;" + #"PORT=3306;" + #"DATABASE=gayakidb;" + #"UID=root;" + #"PASSWORD=mysql;" + #"charset=utf8;";
Next, installed unicode font in the 'Display language setting' of your system OS. Or, you can copy and paste the font file (true type file) in the Fonts folder of Operating System.
Then I set the font property of the textbox object or any other tool object in property page.
More over, if you need to type in unicode font using your keyboard, you need to install the unicode language for keyboard layout. That could be done using the option for regional language settings of your Operating System.
Done. with these settings, i coded module for saving data and while running the ocde, it successfully done the work.