I have a strange UTF8 encoding problem, which I don't understand.
If a friend of mine fills out a form on my webpage, then all german "umlauts" (ä,ü,ö) are displayed in strange chars in my database. When I do the same, they are displayed normally, how it should be. Everything is set to utf8_general_ci, so it should work. But it doesn't, when my friend fills out the form.
Has anyone a suggestion for me?
Thanks!
Even though all tables are UTF-8, the database connection might be using latin-1. What output do you get with SHOW VARIABLES LIKE '%character%'; in MySQL? Any signs of latin-1 there? If so, adjust your charset settings in the MySQL configuration file.
You haven't specified the language you write your app in, and it seems to be connection-based problem. You must manually set connection encoding, f.g. in JDBC, by appending on the end of connection string "?characterEncoding=utf8"
Run SET NAMES utf8 on mysql right after connection
Related
I have imported a database which includes Hebrew. The Hebrew in the database looks like this: קירורית ל 4 פחיות מבית ×מגזית
I tried to change the encoding to UTF-8 but it still looks the same.
How can I fix this?
If any more information needed tell me and I'll provide.
Thanks.
That's "Mojibake" for something close to קירורית ל 4 פחיות מבית �מגזי, correct?
See Trouble with UTF-8 characters; what I see is not what I stored for the cause, to wit:
The bytes to be stored need to be UTF-8-encoded. Fix this.
The connection when INSERTing and SELECTing text needs to specify utf8 or utf8mb4. Fix this.
The column needs to be declared CHARACTER SET utf8 (or utf8mb4). Fix this.
HTML should start with .
The appropriate fix for the data is here: http://mysql.rjweb.org/doc.php/charcoll#fixes_for_various_cases
I've got web-app (jsp) which is using database (mysql). I've put some data into database to test is it work to show in jsp what's in database There was issue with utf-8 characters (polish letters) but i fixed it by adding <parameter-encoding default-charset="UTF-8" /> into glasfish-web.xml. But i still got problem with putting data into database from form. In database instead of polish character i got "?????". I've tried many thinks and nothing Dont reallu now where to look to fixit
ok problem solved I'll put answer for other people having same problem
In jsp where i start my database connection for url="jdbc:mysql://localhost/databasename i changed it into
url="jdbc:mysql://localhost/databasename?useUnicode=true&characterEncoding=UTF-8"
and now everything in database looks like it shoudl
Problem you have faced is that java has escaped the UTF-8 character sequence.
you can use StringEscapeUtils provided by java in order to escape any characters which become ??? or anything else.
try this :
str = org.apache.commons.lang.StringEscapeUtils.unescapeJava(str);
From java
I'm using phpmyadmin for many years but never tried to solve this problem. My database collation is utf8_persian_ci and the table also has the same collation.
While I open phpmyadmin, it doesn't show characters correctly. I see something like برنامه.
In local host I solved this problem by commenting this line:
PMA_DBI_query("SET CHARACTER SET 'utf8';", $link, PMA_DBI_QUERY_STORE);
in /usr/share/phpmyadmin/libraries/database_interface.lib.php but I don't have persmission to do such action in a shared host.
What should I do about this?
NOTE:
The problem is only about PhpMyAdmin. I can see my data correctly when I retrieve them using PHP
To convert your "garbage" data, you need to find out in which character set the data is encoded, then code a PHP application that will read the rows, convert the columns with a function like iconv() (http://www.php.net/manual/en/function.iconv.php) and update the rows.
Hint: your PHP application that displays your data correctly, probably generates a page using a certain character set. So use this character set as the input charset of the iconv() function.
Whenever I try to save ñ it becomes ? in the mysql database. After some few readings it is suggested that I have to change my jsp charset to UTF-8. For some reasons I have to stick to ISO-8859-1. My database table encoding is latin1. How can I fix this? Please help.
Go to your database administration with MySQL WorkBench for example, put the Engine to InnoDB and the collation to utf8-utf8_general_ci.
You state in your question that you require a ISO-8859-1 backend (latin1), and a Unicode (UTF-8) frontend. This setup is crazy, because the set on the frontend is much larger than that allowed in the database. The sanest thing would be using the same encoding through the software stack, but also using Unicode only for storage would make sense.
As you should know, a String is a human concept for a sequence of characters. In computer programs, a String is not that: it can be viewed as a sequence of characters, but it's really a pair data structure: a stream of bytes and an encoding.
Once you understand that passing a String is really passing bytes and a scheme, let's see who sends what:
Browser to HTTP server (usually same encoding as the form page, so UTF-8. The scheme is specified via Content-Type. If missing, the server will pick one based on its own strategy, for example default to ISO-8859-1 or a configuration parameter)
HTTP Server to Java program (it's Java to Java, so the encoding doesn't matter since we pass String objects)
Java client to MySQL server (the Connector/J documentation is quite convoluted - it uses the character_set_server system variable, possibly overridden by the characterEncoding connection parameter)
To understand where the problem lies, first assure that the column is really stored as latin1:
SELECT character_set_name, collation_name
FROM information_schema.columns
WHERE table_schema = :DATABASE
AND table_name = :TABLE
AND column_name = :COLUMN;
Then write the Java string you get from the request to a log file:
logger.info(request.getParameter("word"));
And finally see what actually is in the column:
SELECT HEX(:column) FROM :table
At this point you'll have enough information to understand the problem. If it's really a question mark (and not a replacement character) likely it's MySQL trying to transcode a character from a larger set (let's say Unicode) to a narrower one which doesn't contain it. The strange thing here is that ñ belongs to both ISO-8859-1 (0xF1, decimal 241) and Unicode (U+00F1), so it'd seem like there's a third charset (maybe a codepage?) involved in the round trip.
More information may help (operating system, HTTP server, MySQL version)
Change your db table content encoding to UTF-8
Here's the command for whole DB conversion
ALTER DATABASE db_name DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
And this is for single tables conversion
ALTER TABLE db_table CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
change your table collate to utf8_spanish_ci
where ñ is not equal to n but if you want both characters to be equal use
utf8_general_ci instead
I try several combinations, but this works for me:
VARCHAR(255) BINARY CHARACTER SET utf8 COLLATE utf8_bin
When data retrieve in dbforge express, shows like:
NIÑA
but in the application shows like:
NIÑA
I had the same problem. Found out that is not an issue about encoding UTF-8 or whatever charset. I imported my data from windows ANSI and all my Ñ and ñ where put in the database perfectly as it should be. Example last names showed on database last_name = "MUÑOZ". I was able to select normally from the database with query Select * from database where last_name LIKE "%muñoz%" and phpmyadmin show me results fine. It selected all "MUÑOZ" and "MUNOZ" without a problem. So phpmyadmin does show all my Ñ and ñ without any problems.
The problem was the program itself. All my characters mention, showed as you describe with the funky "MU�OZ" question mark. I had follow all advice everywhere. Set my headers correctly and tried all my charsets available. Even used google fonts and whatsoever font available to display correctly those last names, but no success.
Then I remembered an old program that was able to do the trick back and forth transparently and peeked into the code to figure it out: The database itself, showing all my special characters was the problem. Remember, I uploaded using windows ANSI encoding. Phpmyadmin did as expected, uploaded all as instructed.
The old program fixed this problem translating the Ñ to its UNICODE HTML Entity: Ñ (see chart here https://www.compart.com/en/unicode/U+00D1 ) a process done back and forth from MySQL to the app.
So you just need to change your database strings containing the letter Ñ and ñ to their corresponding UNICODE to reflect correctly on your browser with UTF charset.
In my case, I solved my issues replacing all my Ñ and ñ for their corresponding UNICODE in all the last names in my database.
UPDATE database_name
SET
last_name = REPLACE(last_name,
'MUÑOZ',
'MUÑOZ');
Now, Im able to display, browse, even search all my correct last names and accents/tildes, proper to spanish language. I hope this helps. It was a pain to figure it out, but an old program solved the problem. Best regards and happy coding !
In a very busy PHP script we have a call at the beginning to "Set names utf8" which is setting the character set in which mysql should interpret and send the data back from the server to the client.
http://dev.mysql.com/doc/refman/5.0/en/charset-applications.html
I want to get rid of it so I set default-character-set=utf8 In our server ini file. (see link above)
The setting seems to be working since the relevant server parameters are :
'character_set_client', 'utf8'
'character_set_connection', 'utf8'
'character_set_database', 'latin1'
'character_set_filesystem', 'binary'
'character_set_results', 'utf8'
'character_set_server', 'latin1'
'character_set_system', 'utf8'
But after this change and commenting out set names utf8 call still the data starts to come out garbled.
Please advise....
Setting the encoding of the connection between PHP and MySQL is PHP's job; I don't think the MySQL setting will affect that.
I'd really recommend keeping some code in the application to set the connection character set to UTF-8. The application needs to ensure that the encoding is UTF-8, because it will presumably be telling web browsers its pages are UTF-8, and if those don't match you've got problems.
Since it's already part of the application's responsibility to decide the charset, you might as well make the application the one to specify the database connection character set, rather than leave it as a deployment issue and one more thing to get wrong when installing the app on a new server.
However I would personally use mysql_set_charset to do this rather than SET NAMES.
Similarly, if the application has code in it to create the schema, make sure that code tells MySQL to make the tables UTF-8, rather than leaving it up to the database's default settings and giving yourself another deployment issue to worry about.
skip-character-set-client-handshake
but I don't think that Set names query is too much job. You've got your problem out of nowhere. And going to limit your database functionality