I'm generating cshtml files dynamically for our CMS and using UTF-8 as encoding. I also tried to open those files using Notepad++ and it says that the encoding is UTF-8.
And I just use the controller's View() method to serve the page:
return View(path);
But it still improperly renders the special characters to a wrong one. Like 'α' becoming 'α', or single quote becoming '’'. The generated files when inspecting contains the correct characters, but when it getting served, it shows incorrect characters.
I found the issue and solution. The cshtml files should be written not by simple UTF8 format, but UTF8-BOM file format. Non-BOM UTF8 cshtml files' special characters were converted into something when getting served through return View(path);.
Related
I just used tex4ht and htlatex to convert a latex document into html, and now i have some serious trouble to integrate this html document in a web site i'm making (I'm using Laravel though).I think one of the reason I have some troubles is that htlatex output files are Unix encoded, not utf-8. If I just input the file using Laravel views and controllers without any modification, utf-8 characters are not displayed, and if i convert the file to utf-8, all utf-8 characters turn weird within notepad and I have to rewrite them one at a time (the html files contains 2000+ lines, I can't do that).I'm wondering how can I solve the problem.Is "puting the input html in an iframe" tag an any good solution ? Or is there a way to encode this file to utf-8 without messing with his content ? I'm so lost....
tex4ht uses Latin1 as the default encoding, characters unsupported by this encoding are output as XML entities. You can request UTF-8 output using the following command:
htlatex filename.tex "xhtml,charset=utf-8" " -cunihtf -utf8"
As an alternative, you can use Make4ht with -u option:
make4ht -u filename.tex
make4ht is replacement for htlatex with much more features.
I am working on a Talend Project, Where we are Transforming data from 1000's of XML files to CSV and we are creating CSV file encoding as UTF-8 from Talend itself.
But the issue is that some of the Files are created as UTF-8 and some of them created as ASCII , I am not sure why this is happening The files should always be created as UTF.
As mentioned in the comments, UTF8 is a superset of ASCII. This means that the code point for any ASCII characters will be the same in UTF8 as ASCII.
Any program identifying a file containing only ASCII characters will then simply assume it is ASCII encoded. It is only when you include characters outside of the ASCII character set that the file may be recognised by whatever heuristic the reading program uses.
The only exception to this is for file types that specifically state their encoding. This includes things like (X)HTML and XML which typically start with an encoding declaration.
You can go to the Advanced tab of the tFileOutputDelimited (or other kind of tFileOutxxx) you are using and select UTF-8 encoding.
Here is an image of the advanced tab where to perform the selection
I am quite sure the unix file util makes assumptions based on content of the file being in some range and or having specific start (magic numbers). In your case if you generate a perfectly valid UTF-8 file, but you just use only the ASCII subset the file util will probably flag it as ASCII. In that event you are fine, as you have a valid UTF-8 file. :)
To force talend to get a file as you wish, you can add an additional column to your file (for example in a tMap) and set an UTF-8 character in this column. The generated file will be in UTF8 as the other repliers mentioned.
I ran my web page through the W3C HTML validator and received this error.
The encoding ascii is not the preferred name of the character
encoding in use. The preferred name is us-ascii. (Charmod C024) ✉
Line 5, Column 70: Internal encoding declaration utf-8 disagrees with
the actual encoding of the document (us-ascii).
<meta http-equiv="content-type" content="text/html;charset=utf-8">
Apparently, I am not "actually" using UTF-8 even though I specified UTF-8 in my meta tag.
How do I, well, "actually" use UTF-8? What does that even mean?
The HTML5 mode of the validator treats a mismatch between encoding declarations as an error. In the message, “internal encoding declaration” refers to a meta tag such as <meta charset=utf-8>, and “actual encoding” (misleadingly) refers to encoding declaration in HTTP headers.
According to current HTML specifications (HTML5 is just a draft), the mismatch is not an error, and the HTTP headers win.
There is no real problem if your document only contains Ascii characters. Ascii-encoded data is trivially UTF-8 encoded too, because in UTF-8, any Ascii character is represented as a single byte, with the same value as in Ascii.
It depends on the software used server-side whether and how you can change the HTTP headers. If they now specify charset=ascii, as it seems, it is not a real problem except in validation, provided that you keep using Ascii characters only. But it is somewhat odd and outdated. Try to have the encoding information there changed to charset=utf-8. You need not change the actual encoding, but if you later add non-Ascii characters, make sure you save the file as UTF-8 encoded by selecting a suitable command or option in the authoring program.
Open your file in notepad, then save as > UTF-8 (next to the save button).
On unix-like system you might use iconv tool to convert file from one encoding to another.
It can also be used from the scope of programming language(e.g. php).
The proper function has same name:
http://www.php.net/manual/en/function.iconv.php
Specifying encoding is one thing. Saving documents in a proper encoding is another.
Edit your documents in editors supporting UTF-8 encoding. Preferably UTF-8 without BOM. Notepad++ may be a good start.
Have a read too: UTF-8 all the way through.
I'm working with a web framework that uses a dynamic character encoding in its html templates, like this:
<meta charset="${_response_encoding}">
The problem is when I try to edit this file in Eclipse, Eclipse thinks this is a literal encoding type, and thus refuses to open the file, saying:
"Unsupported Character Encoding" Character encoding
"${_response_encoding}" is not supported by this platform.
Is there any way to tell Eclipse to stop trying to be "smart" (because it plainly isn't) and just show me the text? I've tried using "Open With... Text Editor" but still same result.
Change the content type for HTML files:
Go to Windows -> preferences -> General -> Content types and change encoding (set them to utf-8) for all the file extensions you need.
Choose "Other" and then select UTF-8. Then your template will render as normal.
I had a similar problem, except I was receiving the error message when trying to save the document after changing the character encoding. I resolved the problem by doing the following in Eclipse before putting in the non-standard charset value:
Rename the file to have a non-HTML file extension.
Open the file using an editor other than the HTML one.
Change the charset value to the non-standard value you want.
Rename the file to have the original extension.
Open the file.
Follow the buttons and prompts to set the character encoding to the real encoding of the file.
After this, the file should still be usable while still having the non-standard charset value.
If you're having Eclipse treat it like an HTML file, it is being smart. That's not a valid encoding name. Have you tried just templating the entire meta tag?
(as mentioned in a comment) In Eclipse Indigo, when opening the file you see the Unsupported character encoding message along with a Set Encoding button. Us that button to set the UTF-8 encoding. Eclipse does not change the variable in the HTML file.
True, this is done on a file-by-file basis, however, in my project I import the same meta header file for every screen. Actually, I have only two files to setup (those that are logged in and those that are not).
When i add "default_locale": "en", to my manifest file, then package it and try to upload it, i get the error message saying, An error occurred: Message JSON file must be in UTF-8 encoding. ... But this is how they tell you to set it up in their i18n page... What is the problem?
I'm not sure he's missing the Byte Order Mark -- which is neither required nor recommended in UTF-8 (as endianness isn't relevant) -- I think it's just the file encoding. I believe Notepad adds the BOM by default when you save as UTF-8, but I always save files without it as some programs that receive UTF-8 data don't expect the BOM.
But yes, bottom line, go to the file and Save As in UTF-8. If you are using Notepad, I'd recommend using a different editor -- for JS something lightweight like Notepad++ works well -- so that you can change the default encoding, among many other benefits.
It seems that you simply missing Byte Order Mask in front of your manifest file. If you are on Windows simply open it in Notepad, click File -> Save As and choose UTF-8 from Encoding combo box.
Obviously you can use other tools to "convert" to UTF-8...
just solved the issue... After googling awhile... turns out MS notepad is inadequate even with utf-8 encoding... Download "Notepad2"... and set the encoding to utf-8... somehow it works now!