I'm currently trying to merge data changes between two out of sync expressionengine databases. For this i have opted to use navicat.
The website makes heavy use of the greek character set in templates. When I view greek table field data in phpmyadmin, I see the characters in greek as expected. When I load them up into Navicat, I only see "???" question marks in their place. When I try to sync data between the databases, the result is that question marks are put in the place of greek characters.
The field types in question are "text" using "utf8_general_ci".
what am i doing wrong?
I had the same probem with Navicat Linux... The problem was solved following these steps:
In Navicat, open your table in design view.
Under Fields tab, select the fields which require to show as Unicode. Change the Character set and Collation into utf8 and uft8_general_ci.
Under Options tab, change the Character set and Collation into utf8--UTF-8 Unicode and uft8_general_ci.
Save.
Select Tools > Options. Under Font, change the "Editor Font" into e.g. AR PL ZenKai Uni .
Re-start your Navicat.
Select the correct "Editor Font" in Options until it shows your data correctly. (Repeat steps 5 and 6 above.)
Source: http://wiki.navicat.com/wiki/index.php/How_can_I_display_Unicode_in_Linux%3F
Ran into this issue and saw there is no proper answer.
The solution is:
In Connection Properties... for your MySql database, click on the Advanced tab and check (tick if you're from UK) the box for "Use MySql character set" and then your tables will display correctly.
Under connection's properties, in Advanced tab set encoding to Auto. This is for navicat 15.0.22
Related
I am going crazy. Please help.
I have a form with a text input. This is inserted into My SQL text column. Sometime a user will enter some unknown character that breaks the insert.
For example one that I just found is Topkapı, which is a town in Turkey. You will notice the last character ı. On insert, this causes a database error:
Error Executing Database Query. Incorrect string value: '/xC4/xB1
and...' for column 'country_description' at row 1
Is there a simple method to either remove these characters or escape them? I am using cfqueryparam and tried HTMLEditFormat, CFSavecontent etc to no avail.
EncodeForHTML() does not fix this particular issue if you are actually inserting HTML from TinyMCE for example.
What fixed this was changing the Collation to utf8mb4. You can do this in Workbench by expanding the header. It's collapsed by default.
Backup your table.
Go to "Alter Table".
Click the arrows on the top right of the windows
Select utf8mb4 from the Collation dropdown.
Click "Apply"
Here are your options in my opinion:
If you're using ColdFusion 10 or above, try using EncodeForHTML()
Validate your UI to accept only US and UK English characters, numbers etc.
Change the column data type in MySQL to VARCHAR(n) CHARSET utf8.
Hope this helps.
I'm trying via various DB software (Navicat, MySQL Workbench and others) to include an emoji in the table row along with text. The table is utf8mb4_general_ci, so it should work, but it ends up showing as ???? or ?. What is the trick here to include the emoji icons directly?
The versions of navicat < 11.1 don't work.
Use the newest version of Navicat and go to Edit Connection > Advanced > Encoding and Select 65001 (UTF-8). (This enables to you to insert emojis on the database, try copy/paste from web.whatsapp.com for testing) You will see empty Squares.
Then go to tools > grid font > Segoe UI Symbol (this will show only a utf-7 emojis)
I'm using phpmyadmin for many years but never tried to solve this problem. My database collation is utf8_persian_ci and the table also has the same collation.
While I open phpmyadmin, it doesn't show characters correctly. I see something like برنامه.
In local host I solved this problem by commenting this line:
PMA_DBI_query("SET CHARACTER SET 'utf8';", $link, PMA_DBI_QUERY_STORE);
in /usr/share/phpmyadmin/libraries/database_interface.lib.php but I don't have persmission to do such action in a shared host.
What should I do about this?
NOTE:
The problem is only about PhpMyAdmin. I can see my data correctly when I retrieve them using PHP
To convert your "garbage" data, you need to find out in which character set the data is encoded, then code a PHP application that will read the rows, convert the columns with a function like iconv() (http://www.php.net/manual/en/function.iconv.php) and update the rows.
Hint: your PHP application that displays your data correctly, probably generates a page using a certain character set. So use this character set as the input charset of the iconv() function.
Whenever I try to save ñ it becomes ? in the mysql database. After some few readings it is suggested that I have to change my jsp charset to UTF-8. For some reasons I have to stick to ISO-8859-1. My database table encoding is latin1. How can I fix this? Please help.
Go to your database administration with MySQL WorkBench for example, put the Engine to InnoDB and the collation to utf8-utf8_general_ci.
You state in your question that you require a ISO-8859-1 backend (latin1), and a Unicode (UTF-8) frontend. This setup is crazy, because the set on the frontend is much larger than that allowed in the database. The sanest thing would be using the same encoding through the software stack, but also using Unicode only for storage would make sense.
As you should know, a String is a human concept for a sequence of characters. In computer programs, a String is not that: it can be viewed as a sequence of characters, but it's really a pair data structure: a stream of bytes and an encoding.
Once you understand that passing a String is really passing bytes and a scheme, let's see who sends what:
Browser to HTTP server (usually same encoding as the form page, so UTF-8. The scheme is specified via Content-Type. If missing, the server will pick one based on its own strategy, for example default to ISO-8859-1 or a configuration parameter)
HTTP Server to Java program (it's Java to Java, so the encoding doesn't matter since we pass String objects)
Java client to MySQL server (the Connector/J documentation is quite convoluted - it uses the character_set_server system variable, possibly overridden by the characterEncoding connection parameter)
To understand where the problem lies, first assure that the column is really stored as latin1:
SELECT character_set_name, collation_name
FROM information_schema.columns
WHERE table_schema = :DATABASE
AND table_name = :TABLE
AND column_name = :COLUMN;
Then write the Java string you get from the request to a log file:
logger.info(request.getParameter("word"));
And finally see what actually is in the column:
SELECT HEX(:column) FROM :table
At this point you'll have enough information to understand the problem. If it's really a question mark (and not a replacement character) likely it's MySQL trying to transcode a character from a larger set (let's say Unicode) to a narrower one which doesn't contain it. The strange thing here is that ñ belongs to both ISO-8859-1 (0xF1, decimal 241) and Unicode (U+00F1), so it'd seem like there's a third charset (maybe a codepage?) involved in the round trip.
More information may help (operating system, HTTP server, MySQL version)
Change your db table content encoding to UTF-8
Here's the command for whole DB conversion
ALTER DATABASE db_name DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
And this is for single tables conversion
ALTER TABLE db_table CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
change your table collate to utf8_spanish_ci
where ñ is not equal to n but if you want both characters to be equal use
utf8_general_ci instead
I try several combinations, but this works for me:
VARCHAR(255) BINARY CHARACTER SET utf8 COLLATE utf8_bin
When data retrieve in dbforge express, shows like:
NIÑA
but in the application shows like:
NIÑA
I had the same problem. Found out that is not an issue about encoding UTF-8 or whatever charset. I imported my data from windows ANSI and all my Ñ and ñ where put in the database perfectly as it should be. Example last names showed on database last_name = "MUÑOZ". I was able to select normally from the database with query Select * from database where last_name LIKE "%muñoz%" and phpmyadmin show me results fine. It selected all "MUÑOZ" and "MUNOZ" without a problem. So phpmyadmin does show all my Ñ and ñ without any problems.
The problem was the program itself. All my characters mention, showed as you describe with the funky "MU�OZ" question mark. I had follow all advice everywhere. Set my headers correctly and tried all my charsets available. Even used google fonts and whatsoever font available to display correctly those last names, but no success.
Then I remembered an old program that was able to do the trick back and forth transparently and peeked into the code to figure it out: The database itself, showing all my special characters was the problem. Remember, I uploaded using windows ANSI encoding. Phpmyadmin did as expected, uploaded all as instructed.
The old program fixed this problem translating the Ñ to its UNICODE HTML Entity: Ñ (see chart here https://www.compart.com/en/unicode/U+00D1 ) a process done back and forth from MySQL to the app.
So you just need to change your database strings containing the letter Ñ and ñ to their corresponding UNICODE to reflect correctly on your browser with UTF charset.
In my case, I solved my issues replacing all my Ñ and ñ for their corresponding UNICODE in all the last names in my database.
UPDATE database_name
SET
last_name = REPLACE(last_name,
'MUÑOZ',
'MUÑOZ');
Now, Im able to display, browse, even search all my correct last names and accents/tildes, proper to spanish language. I hope this helps. It was a pain to figure it out, but an old program solved the problem. Best regards and happy coding !
How do I store Unicode in free edition of MySQL?
There doesn't seem to be nvarchar type as in SQL Server. Is Unicode not supported in MySQL? I tried using text but that too is not working.
You need to choose a utf8_* character set for your table. Text and memo fields will then automatically be stored in UTF-8. Support for UTF-16 is coming in mySQL 6.
The character set for a given string column (CHAR, VARCHAR or *TEXT) is determined by its character set and/or collation. This is a quite intense topic so it's best to read the documentation I linked. For example:
CREATE TABLE t1
(
col1 CHAR(10) CHARACTER SET utf8 COLLATE utf8_unicode_ci
)
will create a table t1 with a col1 that stores its content in UTF-8 encoding.
Have you tried setting names after connection? What was the outcome of tryng to store unicode characters? Connect to mysql server and type this:
SET NAMES UTF8;
This should turn on "support" for utf8. Then try storing utf data.
MySQL supports UTF-8.
Use varchar and set encoding for the column (it's quite easy in phpMyAdmin)
http://www.oreillynet.com/onlamp/blog/2006/01/turning_mysql_data_in_latin1_t.html - I think this could help you
Use this Query
alter table `DBNAME`.`TblName` Engine=InnoDB checksum=1 comment='' delay_key_write=1 row_format=dynamic charset=utf8 collate=utf8_unicode_ci
I follow my own style of coding, so please take that into account when following my lead.
First I created the MySql database.
after creating database, in mysql command prompt, give the command:
SET NAMES = UTF8;
table and columns to store unicode are to be set with collation property as utf8-utf_general_ci. Each and every column which meant to store unicode, is to be selected and set collation property to utf8-utf_general_ci.
Now in C#.
my connection string is set as usual but with addition of a single attribute like this:
constring = #"SERVER=localhost;" + #"PORT=3306;" + #"DATABASE=gayakidb;" + #"UID=root;" + #"PASSWORD=mysql;" + #"charset=utf8;";
Next, installed unicode font in the 'Display language setting' of your system OS. Or, you can copy and paste the font file (true type file) in the Fonts folder of Operating System.
Then I set the font property of the textbox object or any other tool object in property page.
More over, if you need to type in unicode font using your keyboard, you need to install the unicode language for keyboard layout. That could be done using the option for regional language settings of your Operating System.
Done. with these settings, i coded module for saving data and while running the ocde, it successfully done the work.