I'm using the UTF-8 charset to write my HTML and some of the text is in Hebrew.
I use the next lines to specify language for browsers:
<html dir="rtl" lang="he">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=9">
Some Explorer browsers which are set on Encoding: Auto-Select recognize my website as Hebrew and view the pages in the Hebrew (Windows) encoding. This makes the text show as gibberish because it is in UTF-8.
How can I use HTML or JavaScript to force all browsers to use the UTF-8 encoding and ignore any local settings that say otherwise?
You cannot override the browser setting of character encoding, if auto-selection is disabled. In such conditions, browsers give the user the final word; this is reflected in the HTML5 draft, which describes, in Determining the character encoding, the first step thusly: “If the user has explicitly instructed the user agent to override the document's character encoding with a specific encoding, optionally return that encoding with the confidence certain and abort these steps.”
The setting Encoding: Auto-Select enables auto-selection, and in your case, this makes browsers apply UTF-8, unless HTTP headers say otherwise. With the given data, it is impossible to say what goes wrong, if you really experience the problem when Auto-Select is on.
Related
I understand that <meta charset="utf-8"> declares the character encoding of the document to be UTF-8.
According to the HTML specs <meta charset="utf-8"> is not mandatory and the only allowed charset for HTML5 is UTF-8 anyways. So leaving it out does not hold back any useful information in an HTML5 document.
Regardless of whether a character encoding declaration is present or not, the actual character encoding used to encode the document must be UTF-8.
However, almost every resource on HTML5 urges to use the tag despite it being redundant and unnecessary with regards to the specs.
Is there any reason to use it?
I have the following html:
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
</head>
<body>
会意字 / 會意字 huìyìzì
</body>
When I run it in firefox, it displays the Chinese characters just fine. How come it works with the ISO-8859-1 characterset? I thought you needed UTF-8?
I can't reproduce your successful rendering:
… but HTML 5 defines a fairly complex character encoding detection method which doesn't pay any attention to <meta> until step 9.
In general, you should avoid encodings other than UTF-8 and definitely should not lie about the encoding of the document.
The most probable explanation is that the document is in fact UTF-8 encoded and the browser treats it that way, despite the meta tag. According to HTML5 encoding sniffing algorithm, which largely reflects browser behavior, the meta tag is ignored if any of the following is true:
The user has instructed (via e.g. a View → Encoding command) the browser to use a specific encoding.
The page starts with bytes that represent the Byte Order Mark in UTF-8 or UTF-16. In practice, it starts that way if the file was saved in an editor with a command like “Save as UTF-8 (with BOM)”.
HTTP headers specify an encoding in a Content-Type header.
You can find out which of these is the cause by using e.g. Rex Swain’s HTTP viewer. It lets you see both the HTTP response headers and the actual data as bytes. Developer Tools in browsers have similar features.
Something that made me curious - supposedly the default character encoding in HTML5 is UTF-8. However if I have a plain simple HTML file with an HTML5 doctype like the code below, I get:
"hello" in Russian: "ЗдраÑтвуйте"
In Chrome 33+, Safari 6, IE11, etc.
<!DOCTYPE html>
<html>
<head></head>
<body>
<p>"hello" in Russian is "здраствуйте"</p>
</body>
</html>
What gives? Shouldn't the browser utilize the UTF-8 unicode standard and display the text correctly? I'm using Coda which is set to save html files with UTF-8 encoding by default so that's not the problem.
The text data in the example is UTF-8 encoded text misinterpreted as window-1252 encoded. The reason is that the encoding has not been specified and browsers are forced to make a guess. To fix this, specify the encoding; see the W3C page Character encodings. Two simple ways that work independently of server settings, as long as the server does not send wrong encoding information in HTTP headers:
1) Save the file as UTF-8 with BOM (there is probably an option for this in your authoring program.
2) Add the following tag into the head part:
<meta charset=utf-8>
There is no single default encoding specified for HTML5. On the contrary, browsers are expected to make guesses when no encoding has been declared. This is a fairly complex process, described in 8.2.2.2 Determining the character encoding.
If you want to be sure which charset will be used by browser you must have in your page head
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
otherwise you are at the mercy of local settings and browser automation.
I wrote html page that displays mixed hebrew/english content.It works fine with charset "windows - 1255"
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN""http://www.w3.org/TR/html4/loose.dtd">
<html dir="rtl" lang="he">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1255">
,But I figured that people will have trouble if their machines doesn't support hebrew. I changed the charset to utf-8 and got
HTML:
meta http-equiv="Content-Type" content="text/html; charset=utf-8"
View:
"��� ��� ������, ��� ����� �����, �� ������ ���� ��� ���� �� ������"
Read zohar ��� ����
....
Isn't utf-8 suppose to support more chars then windows 1255?
I guess when you changed the tag, you didn't tell your editor to convert the file to UTF-8. So, the file is still in Windows-1255 format, but the browser tries to read it as if it was UTF-8, so you get bad/unreadable characters.
I have no idea which editor you're using, so i can't tell you how to put it in UTF-8 mode. Try to find a setting in your options regarding the character set to use. Or, open the file in Windows notepad, and when saving it, make sure you select "Codepage: UTF-8" from the drop down box next to the save button.
Relation to Unicode
The Unicode Hebrew block (U+0590–U+05FF) follows Windows-1255 by encoding both letters and vowel-points in the same relative positions as Windows-1255. Unicode goes further in encoding cantillation marks in lower positions. Unicode Hebrew is always in logical order.
For modern applications UTF-8 or UTF-16 is a preferred encoding.
Source: http://en.wikipedia.org/wiki/Windows_1255
It seems to me that your encoding should still work if your characters are within the Unicode Hebrew block.
I just validated an HTML document using the W3C validator, and found that if I use:
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
with:
<!DOCTYPE HTML>
It throws a warning Line 4, Column 72: Using windows-1252 instead of the declared encoding iso-8859-1.
However, it is fixed if I use:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
I don't really understand what is happening. Also, I don't even know how to use the DOCTYPE tag, I just copied and pasted one from around the web.
Why does this happen?
How should I use the DOCTYPE tag?
Changing the DOCTYPE is simply turning off the warning - it isn't actually fixing anything.
iso-8859-1 and windows-1252 are very similar encodings. They differ only in the characters associated with the 32 byte values from 0x80 to 0x9F, which in iso-8859-1 are mapped to control characters and in windows-1252 are mapped to some useful characters such as the Euro symbol.
The control characters are useless in HTML, and web authors often mistakenly declare iso-8859-1 and yet use one or more of those 32 values as if they were using windows-1252, so browsers when they see the iso-8859-1 charset being declared will automatically change this to be windows-1252.
The validator is simply warning you that this will happen. If you're not using any of the 32 byte values, then you can simply ignore the warning - it's not an error. If you are, and you genuinely want the iso-8859-1 interpretation of the byte values and not the windows-1252 interpretation, you are doing something wrong.
Again, this switching happens in browsers for any DOCTYPE, it's just that the HTML5 validator is being more helpful about what it is telling you than the HTML4 validator is.
A couple of points:
Any HTML5 validation should be taken with a grain of salt. The specification is still under active development, and not everything is set in stone.
You're using the HTML4 syntax for that meta tag. Try <meta charset="iso-8859-1">
That said, HTML validators don't serve that much purpose in this day and age.
But apparently the default for HTML4 was iso=8869-1. That said, the default charset for HTML5 is UTF-8.
More information about the HTML5 doctype can be found in this post by John Resig.
It throws a warning Line 4, Column 72: Using windows-1252 instead of the declared encoding iso-8859-1.
It means the file was saved with the encoding Windows-1252 on creation (AKA Western Windows 1252 or CP1252) and your charset declaration says "hey read this file with ISO 8859-1" when that's not the encoding the file has.
The meta charset exist for that reason. It exist to declare the encoding of the file you are sending/reading/using so when, for example a browser, reads the document it knows what encoding the file is using.
In detail, you have this charset declared:
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
But the file you are validating is actually encoded in Windows-1252. How? Why? Check the text editor you are using and what encoding it is using to save files. If the editor can be configured to change the encoding, choose the one you want to use.
About HTML5
Using
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
or
<meta charset="iso-8859-1">
are both valid for HTML5. See <meta charset="utf-8"> vs <meta http-equiv="Content-Type">
The W3C validator offers options for which encoding the validator uses. You have specified encoding in your document, so you should see "Encoding: iso-8859-1" in the top block of information once the validator has been run.
To the right of that, there is a pull-down menu. Change the choice from "(detect automatically)" to "iso-8859-1 (Western European)". The validator will then use ISO 8859-1 instead of its own choice, and you will not receive the error.
Do the following:
ISO 8859-15. Yeah, -15, and it will work.
Don't place too much stock in the validators. There are typically too many Internet Explorer workarounds, particularly in the CSS content, that will trip up the validator. If your pages work in all browsers and your client is happy, it doesn't matter what some validator says.
If you are specifying the HTML5 doctype, then you should be consistent with the meta charset attribute. Try this though for your pages:
<!DOCTYPE HTML>
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
</body>
</html>