I can't figure this out for the life of me.
I have a query that pulls translations of elements on a page. So any number of 15 languages can appear on that page. When I start to add languages like Swedish anything that has a symbol such as ö results in the whole field returning a null string.
I've verified the encoding on the table and it claims it's using UTF-8 but seeing as how this doesn't work I'm confused.
Here is the query I'm working with:
SELECT
form.form_id,
elem.elem_type,
elem.elem_name,
elem.elem_format,
elem.elem_required,
trans.trans_label,`
trans.trans_description,
trans.trans_defaultValue,
trans.trans_other,
elem.elem_advancedcommand
FROM
events_form form
LEFT JOIN
events_form_elements elem
ON
form.event_id = elem.event_id
INNER JOIN
events_form_translations trans
ON
elem.elem_id = trans.elem_id
INNER JOIN
events_form_languages lang
ON
trans.lang_id = lang.lang_id
WHERE
form.form_id = '{$formid}' AND lang.language = '{$language}'
ORDER BY
elem.elem_sortorder
Now I tried to do something like:
CONVERT(CAST(trans.trans_description as BINARY) USING latin1) as trans_description,
To force it to covert the encoding but that doesn't yield a result at all.
After I get the result it's immediately json_encoded and returned to the user (Ajax Request). I DON'T think it's the json_encode as doing a print_r of the output array yields the same issues.
Also.. lastly, the system I'm building on is using xPDO so I'm not too sure if that's the issue either.
EDIT:
It seems that PHP IS returning a correct value or at least a value for example here is a print_r dump:
[trans_label] => Ditt f�rnamn?
[trans_description] =>
[trans_defaultValue] => First Name
So it seems that when my json_encode touches that string is when it turns the string to null.
Your PDO connection string should specify the encoding. For example:
mysql:host=localhost;port=3306;dbname=test;charset=utf8
This controls the encoding that the database driver will use when it returns a result, and the encoding the driver assumes your queries are in. If you don't specify it, the default encoding will be used. Often the default is latin1.
You can confirm this by printing the hexadecimal representation of the data with bin2hex in PHP: the ö in förnamn is being returned as f6. If the text was encoded in UTF-8 you would obtain c3b6.
You said nothing about the encoding of your web pages.
Do you have that line in the <head> section of your page to force the encoding to UTF-8?
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
I've had problems endcoding letters to my native language Icelandic but ive found a mutual solution for all utf8 letters.
right after mysql_select_db and before mysql_query insert the following:
mysql_query("SET character_set_connection=utf8,
character_set_results=utf8,
character_set_client=utf8", $con);
Where $con is the connection to mysql
Happy coding..
Your answer is null after conversion due to incompatibility of data types.But showing European or Arabic characters on the page is quite simple.I had the same problem with Arabic language, but after few experiment its works fine now.
If you want to show those European characters on the page (jsp,php,html) first set the page encoding to UTF-8 like: -
pageEncoding="utf-8"
And also you need some changes on your database connection class for utf-8 characters
Use the code below:-
jdbc:mysql:your_ipaddress":3306/"+db+"?requireSSL=false&useUnicode=true&characterEncoding=UTF-8
Hope it will help you.
Related
I'm trying to load data from a MySQL DB from a varchar(35) / utf8_swedish_ci field through TBS (tinybutstrong) and PHP using the example (MySQL data merge). My issue is that data loads fine if only ascii characters are in the fields but as soon as I add a single scandinavian special character like ö or ä the field contents vanishes entirely and other fields in row display correctly.
My understanding is that the latest versions on TBS automatically use UTF-8 coding (I have 3.9.0 for PHP 5) so I assumed it would work out-of-the-box. To be safe, I even added the coding to template as so:
'$TBS->LoadTemplate('mysql.html','UTF-8');' but to no avail.
Could someone please advice what is causing this.
For a good UTF-8 processing, all elements of the chain must be UTF-8.
You have to ensure that your template is UTF-8 : check the entered text and the HTML element <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
You have to ensure that all your PHP scripts are UTF-8 and not Ansi.
You also have to ensure that your MySQL connection is set to receive UTF-8 queries and to return UTF-8 item data. This can be done for example by querying the SQL : SET NAMES 'UTF8'
I have stored German umlauts like "ßäöü" in my MySQL database which charset is utf-8.
When I get a value like "Straße" from the DB the output is "Stra�e".
When I display this value with utf8_decode(htmlentities()) the output is completely empty " ".
My html header declares already <meta charset="utf-8">.
How can this issue be solved?
Your database is utf-8. Whats your table and column's encoding? Check on that with phpmyadmin. The output of htmlentities is empty because you are not passing an utf-8 string to it, which defaults to an empty string from php 5.4+ (believe me, that compatibility-breaking decision caused many bugs..)
You could try utf8_decode() if you're using PHP, or any similar function in your language.
I have a wordpress website in which I am using some sort of calendar in multiple languages. In french the special characters ô... are displayed perfectly on the staging server, but are converted to question marks on the live server. Can anyone help point out how to solve this ?
The dates are generated by php:
$langCode = "fr_FR";
setlocale(LC_ALL, $langCode);
$MysqlDate = strtotime(get_post_meta(get_the_ID(), 'date', TRUE));
$month = ucwords(strftime("%b", $MysqlDate));
Also in the head utf-8 is set
<meta charset="utf-8">
setlocale depends on system, specially if you using windows, If you provide a code page like UTF-7 or UTF-8, setlocale will fail, returning NULL.
Here check msdn reference : http://msdn.microsoft.com/en-us/library/x99tb11d.aspx
I want to point you something, that is not very well known: you can add charset info to setlocale().
Example:
Into my utf-8-encoded page I want to insert the name of the current month, which happens to be March, in German "März" - with umlaut. If you use:
setlocale(LC_TIME, 'de_DE');
echo strftime("%B");
...this will return "März", but that html-entity will look like this on a utf-8 page: "M?rz" with question mark. Not what I want.
But if you use:
setlocale(LC_TIME, 'de_DE.UTF8'); // note the charset info !
echo strftime("%B");
...this returns "M√§rz", which, on utf-8, looks like it should: "März".
The same solution will apply to French, just put charset info when calling setlocale().
Because my database is about postage stamps there are a lot of half symbols in the descriptions. They type into mysql with no problem but when I get the results of a query all the half symbols have been replaced by a question mark.
Is there a special code I should be using when I input the descriptions instead of using the half symbol? Otherwise, is there another solution like changing the character set. I'm using utf-8 at the moment.
It seems like you are facing character encoding issue. You should use UTF-8 everywhere:
Make sure the column that contains text data is encoded as utf8_...
Check that when you get information from database, you keep this encoding. You can force it by sending SET NAMES utf8; before any request to MySQL.
Check that when you display this information the encoding is UTF-8 (in a webpage, that means <meta charset='utf-8'> in <head>).
Whenever I try to save ñ it becomes ? in the mysql database. After some few readings it is suggested that I have to change my jsp charset to UTF-8. For some reasons I have to stick to ISO-8859-1. My database table encoding is latin1. How can I fix this? Please help.
Go to your database administration with MySQL WorkBench for example, put the Engine to InnoDB and the collation to utf8-utf8_general_ci.
You state in your question that you require a ISO-8859-1 backend (latin1), and a Unicode (UTF-8) frontend. This setup is crazy, because the set on the frontend is much larger than that allowed in the database. The sanest thing would be using the same encoding through the software stack, but also using Unicode only for storage would make sense.
As you should know, a String is a human concept for a sequence of characters. In computer programs, a String is not that: it can be viewed as a sequence of characters, but it's really a pair data structure: a stream of bytes and an encoding.
Once you understand that passing a String is really passing bytes and a scheme, let's see who sends what:
Browser to HTTP server (usually same encoding as the form page, so UTF-8. The scheme is specified via Content-Type. If missing, the server will pick one based on its own strategy, for example default to ISO-8859-1 or a configuration parameter)
HTTP Server to Java program (it's Java to Java, so the encoding doesn't matter since we pass String objects)
Java client to MySQL server (the Connector/J documentation is quite convoluted - it uses the character_set_server system variable, possibly overridden by the characterEncoding connection parameter)
To understand where the problem lies, first assure that the column is really stored as latin1:
SELECT character_set_name, collation_name
FROM information_schema.columns
WHERE table_schema = :DATABASE
AND table_name = :TABLE
AND column_name = :COLUMN;
Then write the Java string you get from the request to a log file:
logger.info(request.getParameter("word"));
And finally see what actually is in the column:
SELECT HEX(:column) FROM :table
At this point you'll have enough information to understand the problem. If it's really a question mark (and not a replacement character) likely it's MySQL trying to transcode a character from a larger set (let's say Unicode) to a narrower one which doesn't contain it. The strange thing here is that ñ belongs to both ISO-8859-1 (0xF1, decimal 241) and Unicode (U+00F1), so it'd seem like there's a third charset (maybe a codepage?) involved in the round trip.
More information may help (operating system, HTTP server, MySQL version)
Change your db table content encoding to UTF-8
Here's the command for whole DB conversion
ALTER DATABASE db_name DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
And this is for single tables conversion
ALTER TABLE db_table CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
change your table collate to utf8_spanish_ci
where ñ is not equal to n but if you want both characters to be equal use
utf8_general_ci instead
I try several combinations, but this works for me:
VARCHAR(255) BINARY CHARACTER SET utf8 COLLATE utf8_bin
When data retrieve in dbforge express, shows like:
NIÑA
but in the application shows like:
NIÑA
I had the same problem. Found out that is not an issue about encoding UTF-8 or whatever charset. I imported my data from windows ANSI and all my Ñ and ñ where put in the database perfectly as it should be. Example last names showed on database last_name = "MUÑOZ". I was able to select normally from the database with query Select * from database where last_name LIKE "%muñoz%" and phpmyadmin show me results fine. It selected all "MUÑOZ" and "MUNOZ" without a problem. So phpmyadmin does show all my Ñ and ñ without any problems.
The problem was the program itself. All my characters mention, showed as you describe with the funky "MU�OZ" question mark. I had follow all advice everywhere. Set my headers correctly and tried all my charsets available. Even used google fonts and whatsoever font available to display correctly those last names, but no success.
Then I remembered an old program that was able to do the trick back and forth transparently and peeked into the code to figure it out: The database itself, showing all my special characters was the problem. Remember, I uploaded using windows ANSI encoding. Phpmyadmin did as expected, uploaded all as instructed.
The old program fixed this problem translating the Ñ to its UNICODE HTML Entity: Ñ (see chart here https://www.compart.com/en/unicode/U+00D1 ) a process done back and forth from MySQL to the app.
So you just need to change your database strings containing the letter Ñ and ñ to their corresponding UNICODE to reflect correctly on your browser with UTF charset.
In my case, I solved my issues replacing all my Ñ and ñ for their corresponding UNICODE in all the last names in my database.
UPDATE database_name
SET
last_name = REPLACE(last_name,
'MUÑOZ',
'MUÑOZ');
Now, Im able to display, browse, even search all my correct last names and accents/tildes, proper to spanish language. I hope this helps. It was a pain to figure it out, but an old program solved the problem. Best regards and happy coding !