I ran my web page through the W3C HTML validator and received this error.
The encoding ascii is not the preferred name of the character
encoding in use. The preferred name is us-ascii. (Charmod C024) ✉
Line 5, Column 70: Internal encoding declaration utf-8 disagrees with
the actual encoding of the document (us-ascii).
<meta http-equiv="content-type" content="text/html;charset=utf-8">
Apparently, I am not "actually" using UTF-8 even though I specified UTF-8 in my meta tag.
How do I, well, "actually" use UTF-8? What does that even mean?
The HTML5 mode of the validator treats a mismatch between encoding declarations as an error. In the message, “internal encoding declaration” refers to a meta tag such as <meta charset=utf-8>, and “actual encoding” (misleadingly) refers to encoding declaration in HTTP headers.
According to current HTML specifications (HTML5 is just a draft), the mismatch is not an error, and the HTTP headers win.
There is no real problem if your document only contains Ascii characters. Ascii-encoded data is trivially UTF-8 encoded too, because in UTF-8, any Ascii character is represented as a single byte, with the same value as in Ascii.
It depends on the software used server-side whether and how you can change the HTTP headers. If they now specify charset=ascii, as it seems, it is not a real problem except in validation, provided that you keep using Ascii characters only. But it is somewhat odd and outdated. Try to have the encoding information there changed to charset=utf-8. You need not change the actual encoding, but if you later add non-Ascii characters, make sure you save the file as UTF-8 encoded by selecting a suitable command or option in the authoring program.
Open your file in notepad, then save as > UTF-8 (next to the save button).
On unix-like system you might use iconv tool to convert file from one encoding to another.
It can also be used from the scope of programming language(e.g. php).
The proper function has same name:
http://www.php.net/manual/en/function.iconv.php
Specifying encoding is one thing. Saving documents in a proper encoding is another.
Edit your documents in editors supporting UTF-8 encoding. Preferably UTF-8 without BOM. Notepad++ may be a good start.
Have a read too: UTF-8 all the way through.
Related
I recently ran an HTML file I was writing through this on-line HTML validator, and one of the diagnostics I got said,
The character encoding was not declared. Proceeding using
"windows-1252".
When I create a webpage, I write it in a text editor, which saves it as DOS-text (with CR-LF line endings). When I upload the file to my web-hosting provider, it gets converted (I think) on the server to Unix text (LF line endings). My text editor can also save files as Unicode including UTF-8, but I rarely find that necessary.
The standard online advice about specifying the character encoding in a web document is to include, just under the <head> tag, <meta charset="utf-8">. There is also advice that you should ensure that what you specify does not conflict with the information sent by the server in the HTTP headers when serving the document. Using Rex Swain's [online] HTTP viewer, I see that in the HTTP headers it just says,
Content-Type:·text/html
Should I follow the standard advice to specify the charset as UTF-8, even though the html file is never saved as such, or should I specify it as windows-1252, as assumed by that online validator, or as ISO-8859-1 as per one of the example values on W3Schools? Also, some examples of the charset metatag show it terminated as />. Which is the preferred syntax, and should there be a space before the slash?
I am working on a Talend Project, Where we are Transforming data from 1000's of XML files to CSV and we are creating CSV file encoding as UTF-8 from Talend itself.
But the issue is that some of the Files are created as UTF-8 and some of them created as ASCII , I am not sure why this is happening The files should always be created as UTF.
As mentioned in the comments, UTF8 is a superset of ASCII. This means that the code point for any ASCII characters will be the same in UTF8 as ASCII.
Any program identifying a file containing only ASCII characters will then simply assume it is ASCII encoded. It is only when you include characters outside of the ASCII character set that the file may be recognised by whatever heuristic the reading program uses.
The only exception to this is for file types that specifically state their encoding. This includes things like (X)HTML and XML which typically start with an encoding declaration.
You can go to the Advanced tab of the tFileOutputDelimited (or other kind of tFileOutxxx) you are using and select UTF-8 encoding.
Here is an image of the advanced tab where to perform the selection
I am quite sure the unix file util makes assumptions based on content of the file being in some range and or having specific start (magic numbers). In your case if you generate a perfectly valid UTF-8 file, but you just use only the ASCII subset the file util will probably flag it as ASCII. In that event you are fine, as you have a valid UTF-8 file. :)
To force talend to get a file as you wish, you can add an additional column to your file (for example in a tMap) and set an UTF-8 character in this column. The generated file will be in UTF8 as the other repliers mentioned.
My application stores configuration data (including strings for the UI) in a text file containing JSON. For example, config.json might contain the following:
{
"CustomerName" : "Omni Consumer Products",
"SubmitButtonText": "Click here to submit",
// etc etc etc..
}
This file goes to our translation vendor, who makes duplicates of it in multiple supported languages. They might be building their own app, or they might be editing it in a text editor. I don't know.
Since we're going to be using all manner of non-ASCII characters in some of our languages, I'd like to ensure everybody is clear on what character encoding we're using.
So if this were an XML file, I would stick the following declaration at the top of the file:
<?xml version="1.0" encoding="UTF-8"?>
Any reasonable text editor or XML parser will see this and know that the file is encoded in UTF-8.
Is there any similar standard I can put at the top of a JSON file, and be reasonably assured that consumers will play nicely with it?
JSON's default encoding is UTF-8:
http://www.ietf.org/rfc/rfc4627.txt
From section 3:
JSON text SHALL be encoded in Unicode. The default encoding is UTF-8.
Since the first two characters of a JSON text will always be ASCII characters [RFC0020], it is possible to determine whether an octet stream is UTF-8, UTF-16 (BE or LE), or UTF-32 (BE or LE) by looking at the pattern of nulls in the first four octets.
This determination is unambiguous so there is no special place where an encoding is described in the format itself.
I'm working with a web framework that uses a dynamic character encoding in its html templates, like this:
<meta charset="${_response_encoding}">
The problem is when I try to edit this file in Eclipse, Eclipse thinks this is a literal encoding type, and thus refuses to open the file, saying:
"Unsupported Character Encoding" Character encoding
"${_response_encoding}" is not supported by this platform.
Is there any way to tell Eclipse to stop trying to be "smart" (because it plainly isn't) and just show me the text? I've tried using "Open With... Text Editor" but still same result.
Change the content type for HTML files:
Go to Windows -> preferences -> General -> Content types and change encoding (set them to utf-8) for all the file extensions you need.
Choose "Other" and then select UTF-8. Then your template will render as normal.
I had a similar problem, except I was receiving the error message when trying to save the document after changing the character encoding. I resolved the problem by doing the following in Eclipse before putting in the non-standard charset value:
Rename the file to have a non-HTML file extension.
Open the file using an editor other than the HTML one.
Change the charset value to the non-standard value you want.
Rename the file to have the original extension.
Open the file.
Follow the buttons and prompts to set the character encoding to the real encoding of the file.
After this, the file should still be usable while still having the non-standard charset value.
If you're having Eclipse treat it like an HTML file, it is being smart. That's not a valid encoding name. Have you tried just templating the entire meta tag?
(as mentioned in a comment) In Eclipse Indigo, when opening the file you see the Unsupported character encoding message along with a Set Encoding button. Us that button to set the UTF-8 encoding. Eclipse does not change the variable in the HTML file.
True, this is done on a file-by-file basis, however, in my project I import the same meta header file for every screen. Actually, I have only two files to setup (those that are logged in and those that are not).
Suppose I have an input field in a web page with charset UTF8; suppose I open a text file encoded with ISO-8859-1 as charset.
Now I copy and paste a string with special characters (like, for example, ô) from file to the input field : I see that the special characters is correctly displayed into input field.
Who does the conversion from ISO-8859-1 to UTF8? The browser?
When you open the file and copy/paste it to the browser, it ends up in Unicode, as that is what the browser's UI controls use internally. Who actually performs the conversion from ISO-8859-1 to Unicode depends on a few factors (what OS you are using, whether your chosen text editor is compiled to use Ansi or Unicode, what clipboard format(s) - CF_TEXT for Ansi, CF_UNICODETEXT for Unicode - the app uses for the copy, etc). But either way, when the web browser submits the form, it then encodes its Unicode data to the charset of the HTML/form during transmission.
In all likelihood, it's not really converted to UTF-8, but instead to the internal representation of characters used by the browser, which is quite likely to be UTF-16 (no matter what the encoding of the web page is).