I have a string like 'دبي' and i want to get its correct unicode character. Currently, I am using str.charCodeAt(index) to get its unicode character but for Arabic characters it gives between 0600 and 06FF. However, i want Arabic Presentation Forms A and B - whichever is actually written.
Can anyone suggest how to do this?
The string you posted consists of three normal Arabic letters in the 0600...06FF range, so what you are getting is the correct Unicode characters. If you mean that you would like to determine the contextual glyph forms used, then that’s outside the character level and cannot be determined from the string. (It can be determined, by applying rules of Arabic writing, which forms should be used, but that’s different from knowing which forms are actually used by the rendering software.)
Arabic Presentation Forms are legacy characters not meant for normal use. Normal rendering is not supposed to convert normal character to such forms but to select glyphs contextually.
Related
I have a web application with embedded fonts. There is a small problem. Language of my web application is Persian and English but all numbers in the web page are shown in Persian even the numbers in the English content. This is the screenshot of web application.
Is there any way to show numbers like Microsoft Word (use Persian numbers in Persian text and English numbers in English text)?
Technically, you could put both common European digits 0, 1, 2… and Arabic digits ٠, ١, ٢, … as alternative glyphs for the characters U+0030 DIGIT ZERO, U+0031 DIGIT ONE, U+0032 DIGIT TWO, etc.,into the same font, using OpenType features, and you could use CSS tools for selecting between (though this is not yet supported by all browsers). But then you would need to be a font designer, or at least know how to edit a font.
The normal way, however, is to treat European digits and Arabic digits as distinct characters, i.e. make the difference at the character level. So the code that generates the calendar should take care of the issue. And then you just need a font that has both sets of digits, properly assigned to the separate characters.
Its quite easy if you use lang attribute inside the parent div of your calendar:
<div lang="en" style="font-family:Tahoma">
<!--Calendar code here -->
</div>
I want to take characters in the Microsoft Symbol font (taken from the w:sym tag in a docx file) and render them in html. When I look at how Word writes out the characters when I save the doc as html, I see this:
<span style='mso-char-type:symbol;mso-symbol-font-family:Symbol'>Â</span>
This appears as a script R in both Word and Word's html output.
When I write the same thing in my own html file, I see the A-hat in the regular font, and Chrome's element inspector warns that the mso- properties are unknown.
In Word's html output there is lots of mso-specific stuff but nothing I can see that lets Chrome know how to interpret mso-char-type and mso-symbol-font. I see the same behavior in IE.
Is there an easy way to tell the browser to use the Symbol font? Or do I have to explicitly translate the Symbol font characters to Unicode (using a static translation table?)
Thanks,
Wayne
The Symbol font is a privately-encoded font, i.e. it places various glyphs in positions that should be occupied by other characters according to character code standards. This means that a web page using it will fail badly whenever the Symbol font is not available, or the page style sheet is overriden, or the browser behaves correctly: e.g., the letter “” cannot be rendered using the Symbol font, so the browser will use a fallback font.
The proper way is to use Unicode encoded characters, such as “ℜ”, in a UTF-8 encoded page, with font-family on the applicable element set to contain a list of fonts that contain this character. For general notes on this, see my Guide to using special characters in HTML.
An inappropriate way that has worked on some faulty browsers is to set font to Symbol in a manner generally understood by browsers, e.g. <font face=Symbol>Â</font> or <span style="font-family: Symbol">Â</span>. But as said, if this “works”, consider it a browser bug.
So yes, if you now have data using Symbol font, it should be mapped to Unicode characters.
Note that characters like “ℜ” (Black-letter capital R, not script R) are seldom needed. In particular, the standard (as per ISO 80000-2) notation for the real part of a complex number z is not ℜ(z) but Re z.
Ok, just removing mso-symbol and writing font-family:Symbol seems to have worked. However I suspect this is not really best practice... A table for translating symbols into unicode can be found here: http://www.alanwood.net/demos/symbol.html
I am building a website for a German client, so the text on the website will regularly contain characters like:
ä
ö
ü
ß
Is it necessary for to convert all those characters to their HTML Entities while the website uses UTF-8 character encoding everywhere?
Or maybe there's no relation between the two areas?
When (if at all) should I convert those to their HTML Entities, then?
You should convert to HTML entity or character references when:
a. you are stuck with some editor or processing component that doesn't support Unicode properly;
b. you have manually-edited markup with confusable characters. For example, if you have a non-breaking-space that is important to lay out correctly, you might want to write it as or so that it's obvious and doesn't get replaced with a normal space when someone edits the file.
Other than that, no, just go with the raw versions.
I want to insert the Arabic letters in the pattern just like the English letters
pattern="[a-zA-Z0-9-_. ]{1,30}"
I have no idea how to accomplish the action.
The range for Arabic and Persian are shared so this code could be used for Arabic too.
[أ-يa-zA-Z]
This is the reference for finding the character range of Unicode languages:
preg_replace and preg_match arabic characters
http://unicode.org/charts/
The HTML5 pattern attribute follows JavaScript regular expression syntax, which makes things rather awkward. You cannot test character properties, for example. Instead, you need to list down the allowed characters or ranges of characters.
Using the normative Scripts.txt file (by the Unicode Consortium), which defines the script (writing system) of all characters, I constructed the following:
pattern=
"[a-zA-Z0-9-_. \
\u0620-\u063F\u0641-\u064A\u066E-\u066F\u0671-\u06D3\u06D5\
\u06E5-\u06E6\u06EE-\u06EF\u06FA-\u06FC\u06FF\u0750-\u077F\
\u08A0\u08A2-\u08AC\uFB50-\uFBB1\uFBD3-\uFD3D\uFD50-\uFD8F\
\uFD92-\uFDC7\uFDF0-\uFDFB\uFE70-\uFE74\uFE76-\uFEFC]{1,30}"
Starting from the set of all characters with script defined to be Arabic, I picked up those that are declared as letters (General Category Lo or Lm), and then omitted those beyond BMP, the Basic Multilingual Plane.
Characters outside BMP are used very rarely, and to represent them in JavaScript syntax, you would need to either include the characters themselves or use two \u notations per character (one for each component of a surrogate pair). This does not sound realistic.
This is of course a “hardwired” solution: it may need updates if new Arabic letters are added to Unicode or the script of a character is changed from or to Arabic (which is highly unlikely). But I don’t expect to see new Arabic letters added to BMP during my lifetime.
When I copy/paste text from most sites and pdfs, the following characters are almost always in the unicode equivalent:
double quote: " is “ and ” (“ and ”)
single quote: ' is ‘ and ’ (‘ and ’)
ellipsis: ... is … (…)
I understand ones that can't be represented without unicode like © and ¢, but even for those, I wonder.
When should you use these unicode equivalents? Are they more semantic than not using them? Are they better interpreted by devices (copy/paste/print)? I always find it annoying getting those quote and ellipsis characters because with textmate + programming, you don't use them.
When should you use these unicode equivalents? Are they more semantic than not using them?
Note that these are not “unicode equivalents”. Those characters are available in many character sets other than Unicode, and they are strictly distinct from the alternatives that you propose.
In typography, the left and right versions of the single and double quotation marks are correct. They provide the traditional appearance for those characters that has been used in print media for many years. The ellipsis character provides the correct spacing for an ellipsis that does not naturally occur when using consecutive full stop characters. So the reason all of these are used is to make the text appear correctly to human readers.
Are they better interpreted by devices (copy/paste/print)?
Any system that uses any character set should be designed to correctly handle that character set. If the text is encoded in Unicode, then any recent system (from the last 15 years at least) should be able to handle it, since Unicode is the de facto standard character set for all modern systems.
Not all Unicode-conformant systems will be able to display all characters correctly. This will depend on the fonts available, and even the rendering system that uses the fonts. But any Unicode-conformant system will be able to transmit the characters unaltered (such as in a copy and paste operation).
I always find it annoying getting those quote and ellipsis characters because with textmate + programming, you don't use them.
It is unusual to copy English (or whatever language) text directly into a program without having to add separate delimiters to that text. But most modern programming languages will not have any difficulty handling the text once it is property delimited.
Any systems that cannot handle Unicode correctly should be updated. Legacy character encodings will have no place in the future.
I think there's a simple explanation: MS Word converts these characters/sequences automatically as you type and a lot of text in the internet has been copied from this text editor.
Most of the articles I get for my site from other authors are sent as .doc file and I have to convert it. Usually, it contains these characters you've mentioned.
I'd also add one more: many different types of dashes instead of the hyphen. And also the low opening double quote (as seen in some european languages).
I usually let them stay in the text (all my pages are unicode). It's just important to remember it when playing around with regex etc (especially the dashes can be tricky and hard to spot).
HTML entities serve a triple purpose:
Being able to use characters that do not belong to the document character set, e.g., insert an euro symbol in a ISO-8859-1 document.
Escape characters that have a special meaning in HTML, such as angle brackets.
Make it easier to type characters that are not in your keyboard or are not supported by your editor, e.g. a copyright symbol.
Update:
My info is correct but I suspect I've answered the wrong question...
On the web, I would consider that markup adds semantic meaning, content does not. So it doesn't really matter which you use in this context.
Typographers would insist on “ and ”, where programmers don't care and just use regular old quotes ".
The key here is interoperability. There are different encoding schemes. As we've all been victim to, people paste content into an editor from WORD, which uses windows-1251 encoding. When you serve this content up via AJAX is usually breaks because AJAX uses UTF-8 encoding by default.
Office 2010 now allows for the saving of documents in UTF-8 format. Also, databases have different unicode encoding schemes. The best bet is to use UTF-8 end-to-end.
When you copy-pasta text that includes special characters, they will be left as they are. This is perfectly fine if the characters match the charset used by the webpage.
HTML entities are just a convenience for producing specific characters in any character set. Keyboards tend not to have keys to get symbols like ©, so the HTML entity is a shortcut.
I'm going to generalize and say that most of the time the content is UTF-8 (please correct me if I'm wrong). The copied characters are usually copied correctly and everything works great, if they aren't copied correctly, or the charset is subject to change, or you're after i18n support, go with the HTML or XML entities. Otherwise, leave them as they are, the browser will display them just fine.