Hello I am trying to compile an EPUB v2.0 with html code extracted from Indesign. I have noticed there are a lot of "special characters" either at the beginning of a paragraph or at the end. For example
<p class="text_indent0px font_size0_8em line_height1_325 margin_bottom1px margin_left0px margin_right0px sans_serif floatleft">E<span class="small_caps">VELYNE</span> </p>
What is this
and can I either get rid of it or replace it with a "nbsp;"?
	
Is the ascii code for tabs. So I guess the paragraphs were indented with tabs.
If you want to replace them with then use 4 of them
That would be a horizontal tab (i.e. the same as using the tab key).
If you want to replace it, I would suggest doing a find/replace using an ePub editor like Sigil (http://sigil-ebook.com/).
represents the horizontal tab
Similarly represent space.
To replace you have to use
In the HTML encoding &#{number}, {number} is the ascii code. Therefore, is a tab which typically condenses down to one space in HTML, unless you use CSS (or the <pre> tag) to treat it as pre formatted text.
Therefore, it's not safe to replace it with a non-breaking or a regular space unless you can guarantee that it's not being displayed as a tab anywhere.
div:first-child {
white-space: pre;
}
<div> Test</div>
<div> Test</div>
<pre> Test</pre>
See https://developer.mozilla.org/en-US/docs/Web/CSS/white-space and http://ascii.cl/
is the entity used to represent a non-breaking space
decimal char code of space what we enter using keyboard spacebar
decimal char code of horizontal tab
and both represent space but is non-breaking means multiple sequential occurrence will not be collapsed into one where as for the same case, ` will collapse to one space
= approx. 4 spaces and approx. 8 spaces
There are four types of character reference scheme used.
Using decimal character codes (regex-pattern: &#[0-9]+;),
Using hexadecimal character codes (regex-pattern: &#x[a-f0-9]+;),
Using named character codes (regex-pattern: &[a-z]+;),
Using the actual characters (regex-pattern: .).
Al these conversions are rendered same way. But, the coding style is different. For example, if you need to display a latin small letter E with diaeresis then you could use any of the below convention:
ë (decimal notation),
ë (hexadecimal notation),
ë (html notation),
ë (actual character),
Likewise, as you said, what should be used (a) (decimal notation) or (b) (html notation) or (c) (decimal notation).
So, from the above analogy, it can be said that the (a), (b) and (c) are three different kind of notation of three different characters.
And, this is for your information that, (a) is a Horizontal Tab, the (b) one is the non-breaking space which is actually in decimal notation and the (c) is the decimal notation for normal space character.
Now, technically space at the end of the paragraph, is nothing but meaningless. Better, you could discard those all. And if you still need to use space inside <pre> elements, not in <p> or <div>.
Hope this helps...
Related
The Unicode catalogue includes a number of white-space characters, some of which don't appear to work in any context in HTML documents - but some of which, rather usefully, do.
Here is an example:
<h1 title="Hi! As a title attribute,
I can contain horizontal tabs
and carriage returns
and line feeds.">HTML's handling of &009; | &010; | &013;</h1>
<p>Hello. As a paragraph element, I can't contain horizontal tabs
or carriage returns
or line feeds.</p>
<input type="submit" value="I am a value attribute and
like title I can also handle line feeds" /><br />
<input type="submit" value="I am another value attribute. Like title I can handle horizontal tabs" /><br />
<input type="submit" value="I am a third value attribute.
Unlike title I can't handle carriage returns" />
Is there any official spec or series of guidelines which detail which white-space characters can be deployed in HTML documents and where?
It's a little unclear what you mean by work, but I'm going to assume you mean rendering, at which point what happens is really up to CSS.
https://www.w3.org/TR/CSS2/text.html#white-space-model defines how most whitespace characters are normalized away, unless you adjust the white-space property.
Note that the display of toolbars (such as from the title attribute) and form controls (such as from input elements) is not defined by any standard, leaving that effectively up to browsers.
Disclaimer: this answer was composed for the question as originally written, making explicit references to ASCII control characters. It was apparently a red herring so the information here may look confusing now.
First of all, I don't think nobody uses ASCII any more. In 2016 the only sensible encoding is UTF-8. Whatever, UTF-8 is a superset of ASCII (and you can use ASCII anyway) so the question is still be valid.
Secondly, your example isn't correct. All the HTML entities you mention are printable characters:
is 'CHARACTER TABULATION' (U+0009) (i.e. a tab)
is 'CARRIAGE RETURN (CR)' (U+000D) (i.e. a legacy MacOS line feed)
is 'LINE FEED (LF)' (U+000A) (i.e. a Unix line feed)
(And please note that Windows line feeds are a combination of CR+LF.)
If you're really talking about control characters:
EOT End of Transmission
ACK Acknowledgement
BEL Bell
...
... we first need to understand that HTML is meant to be plain text (as such, it's MIME content type is text/html). The HTML5 Living Standard provides a definition of control character that's wider than the ASCII one but in any case it doesn't seem to be allowed:
Any occurrences of any characters in the ranges U+0001 to U+0008,
U+000E to U+001F, U+007F to U+009F, U+FDD0 to U+FDEF, and characters
U+000B, U+FFFE, U+FFFF, U+1FFFE, U+1FFFF, U+2FFFE, U+2FFFF, U+3FFFE,
U+3FFFF, U+4FFFE, U+4FFFF, U+5FFFE, U+5FFFF, U+6FFFE, U+6FFFF,
U+7FFFE, U+7FFFF, U+8FFFE, U+8FFFF, U+9FFFE, U+9FFFF, U+AFFFE,
U+AFFFF, U+BFFFE, U+BFFFF, U+CFFFE, U+CFFFF, U+DFFFE, U+DFFFF,
U+EFFFE, U+EFFFF, U+FFFFE, U+FFFFF, U+10FFFE, and U+10FFFF are parse
errors. These are all control characters or permanently undefined
Unicode characters (noncharacters).
Any character that is a not a Unicode character, i.e. any isolated
surrogate, is a parse error. (These can only find their way into the
input stream via script APIs such as document.write().)
If you actually refer to the characters in your example, some of then are considered exceptions in the parsing stage:
U+000D CARRIAGE RETURN (CR) characters and U+000A LINE FEED (LF)
characters are treated specially. Any LF character that immediately
follows a CR character must be ignored, and all CR characters must
then be converted to LF characters. Thus, newlines in HTML DOMs are
represented by LF characters, and there are never any CR characters in
the input to the tokenization stage.
... but I suspect you are only interested in white-space collapsing:
In HTML, only the following characters are defined as white space
characters:
ASCII space ( )
ASCII tab ( )
ASCII form feed ()
Zero-width space ()
[...]
In particular, user agents should collapse input white space sequences
when producing output inter-word space.
[...]
The PRE element is used for preformatted text, where white space is
significant.
In other words, consecutive white space characters become a simple space (except inside <pre> tag). (I could only find a link for HTML 4 but that's something that hasn't changed significantly).
Is there any official spec or series of guidelines? Sure they are: you have the official W3C recommendations and the WHATWG specs but they're basically technical documentation mostly addressed at browser vendors: extensive, comprehensive and hard to decipher into plain English ;-)
In my html page I have displayed fractions using html special character. My idea is to display 1/2, 2/2 and 3/3.
I have used ⅓ for 1/3 and ⅔ for 2/3 and the special charactera are displayed correctly. I took reference from this link HTML Special Characters
But when I tried using &frac33; for 3/3 it is not working. It is just displaying as it is, not converting to special character.
Could you someone please tell me what is the html special character for 3/3.
Thank You
<sup>3</sup>⁄<sub>3</sub>
Result: 3⁄3
Not all fractions have their own special character. For those fractions (like 3/3) which don't have slanted fraction characters, use the HTML entity ⁄:
<sup>3</sup>⁄<sub>3</sub> = 3⁄3
There is no named (or numeric) character reference for a character representing 3/3, since there simply is no such character.
In theory, the FRACTION SLASH U+2044 “⁄” character (representable as ⁄ in HTML, among other thing) can be used between digits to suggest that rendering routines present the combination as a typographic fraction. In practice, only some typesetting programs can do this, and web browsers come nowhere near.
Trying to play with HTML markup and/or CSS to construct something that looks like a typographic fraction (comparable to ½ in appearance) tend to produce messy results, including uneven line spacing.
The practical option is to use just common notations like 2/2. But if you want something like a typographic fraction, you could use MathML with MathJax. More exactly, you would use the mfrac element in MathML with the attribute bevelled="true". Sample code:
<!doctype html>
<title>Fractions with MathJax and MathML</title>
<script src=
"http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
Here we have the common fraction ½, then
a simulation with HTML and CSS:
<sup>1</sup>⁄<sub>2</sub>.
Note that this tends to create uneven line spacing.
There are some cures to that, but let us see how MathML works:
<math>
<mfrac bevelled="true">
<mn>1</mn>
<mn>2</mn>
</mfrac>
</math>.
Some text here to demonstrate that line spacing has not
been disturbed here.
Sample rendering:
From wikipedia:
Cursive (from Latin curro, currere, cucurri, cursum, to run, hasten) is any style of handwriting that is designed for writing notes and letters quickly by hand. In the Arabic, Latin, and Cyrillic writing systems, the letters in a word are connected, making a word one single complex stroke.
In the above languages when we want to format one single word with e.g. <span> tag to apply custom css style it breaks word conection, so is there any solution for this.
example this is for example normal arabic word: كتب
but when we want to color last letter in other color using the span tag get this:
because first two letter are in one tag and last is in other to color it.
Is there something I can do to avoid word breaks.
Here is the full html:
<p>كت<span style="color: Red;">ب</span></p>
I'm not sure if there's any HTML way to do it, but you can fix it by adding a zero-width joiner Unicode character before the opening span tag:
<p>كت<span style="color: Red;">ب</span></p>
You can use the actual Unicode character instead of the HTML character entity, of course, but that wouldn't be visible here. Or you can use the prettier entity.
Here it is in action (using an invisible <b> tag, since I can't do color here), without the joiner:
كتب
and with the joiner:
كتب
It's supposed to work without the joiner as far as I understand it, though, and it does in some browsers, but clearly not all of them.
Update 2020/5
Google Chrome (Checked version 81.0.4044.138) and Firefox (76.0.1) have solved this issue when rendreing Arabic and Farsi words and there is no more need to handle the situation manually. Simply wrap the keyword with <span style="color:red">Keyword</span> works fine with both connecting and non-connecting characters.
For this reason, you probably can not see the difference between Correct and Wrong examples below:
Main post:
After 7 years of accepted answer I would like to add a new answer with more practical details as my native language is Farsi. I assume that we want to replace a keyword within a long word. This answer considers the following details:
1- Sometimes it is not enough to add only to the previous character becase next character should also has a tail to complete the connection.
body{font-size:36pt;}
span{color:red}
Wrong: مک<span>انیک</span>
<br>
Correct: مک<span>انیک</span>
2- We may also need to add after the keyword to connect it to next character.
body{font-size:36pt;}
span{color:red}
Wrong: مک<span>انیک</span>ی
<br>
Correct: مک<span>انیک</span>ی
3- There are some characters that accept tail before but not after. So we have to exclude them from accepting tail after them. This is the list of non-connecting characters to next characters: ا آ د ذ ر ز ژ و
4- Finally to respect search engines and scrappers, I recommend using javascript (jquery) to replace keywords after DOM ready to keep the page source clean.
This is my final code with regards to all details above:
$(document).ready(function(){
var tail="\u200D";
var keyword="ستر";
$(".searchableContent").each(function(){
var htm=$(this).html();
/*
preserve keywords which have space both before and after
with a temp sign say #fullHolder#
*/
htm=htm.split(' '+keyword+' ').join(' #fullHolder# ');
/*
preserve keywords which have only space after
with a temp sign say #preHolder#
*/
htm=htm.split(keyword+' ').join('#preHolder#'+' ');
/*
preserve keywords which have only space before
with a temp sign say #nextHolder#
*/
htm=htm.split(' '+keyword).join(' '+'#nextHolder#');
/*
replace remaining keywords with marked up span.
Add tail to both side of span to make sure it is
connected to both letters before and after
*/
htm=htm.split(keyword).join(tail+'<span style="color:#ff0000">'+tail+keyword+tail+'</span>'+tail);
//Deal #preHolder# by adding tail only before the keyword
htm=htm.split('#preHolder#'+' ').join(tail+'<span style="color:#ff0000">'+tail+keyword+'</span>'+' ');
//Deal #nextHolder# by adding tail only after the keyword
htm=htm.split(' '+'#nextHolder#').join(' '+'<span style="color:#ff0000">'+keyword+tail+'</span>'+tail);
//Deal #fullHolder# by adding markup only without tail
htm=htm.split(' '+'#fullHolder#'+' ').join(' '+'<span style="color:#ff0000">'+keyword+'</span>'+' ');
//Remove all possible combination of added tails to non-connecting characters
var nonConnectings=['ا','آ','د','ذ','ر','ز','ژ','و'];
for (x = 0; x < nonConnectings.length; x++) {
htm=htm.split(nonConnectings[x]+tail).join(nonConnectings[x]);
htm=htm.split(nonConnectings[x]+'<span style="color:#ff0000">'+tail).join(nonConnectings[x]+'<span style="color:#ff0000">');
htm=htm.split(nonConnectings[x]+'</span>'+tail).join(nonConnectings[x]+'</span>');
}
$(this).html(htm);
})
})
div{font-size:26pt}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<div class="searchableContent">
سترون - بستری - آستر - بستر - استراحت
</div>
I've two questions:
If I take out any text using text() or as_trimmed_text() function and want to push in some element then do I need to use HTML::Entities::encode_entities? :
my $text=$node->as_trimmed_text();
$a->push_content($text); # Do I need to use encode_entities here?
Secondly after processing and generating whole html document using as_HTML() it's sometimes generating some special characters for example: Â(Â) as an extra char when all I see is single space in Dreamweaver.
I have two answers:
Assuming that you want the content of $a to be the same as the content of $node, you do not need to encode_entities as push_content inserts the passed string as a text node rather than parsing it as markup. OTOH, if the content of $node is <span> (represented in HTML source as <span>) and you actually want $a to display <span> (represented in HTML source as <span>), you would call encode_entities on it.
Chances are that your input text contains raw UTF-8 characters which the code is interpreting as Latin-1 or a similar encoding. The "single space" characters are actually U+00A0, non-breaking space, which is represented in UTF-8 by the two bytes 0xc2 0xa0, which when interpreted in Latin-1 are "Â" and non-breaking space.
Why indeed? Wouldn't something like &br; be more appropriate?
An HTML entity reference is, depending on HTML version either an SGML entity or an XML entity (HTML inherits entities from the underlying technology). Entities are a way of inserting chunks of content defined elsewhere into the document.
All HTML entities are single-character entities, and are hence basically the same as character references (technically they are different to character references, but as there are no multi-character entities defined, the distinction has no impact on HTML).
When an HTML processor sees, for example — it replaces it with the content of that entity reference with the appropriate entity, based on the section in the DTD that says:
<!ENTITY mdash CDATA "—" -- em dash, U+2014 ISOpub -->
So it replaces the entity reference with the entity — which is in turn a character reference that gets replaced by the character — (U+2014). In reality unless you are doing this with a general-purpose XML or SGML processor that doesn't understand HTML directly, this will really be done in one step.
Now, what would we replace your hypothetical &br; with to cause a line-break to happen? We can't do so with a newline character, or even the lesser known U+2028 LINE SEPARATOR (which semantically in plain text has the same meaning as <br/> in HTML), because they are whitespace characters which are not significant in most HTML code, which is something that you should be grateful for as writing HTML would be much harder if we couldn't format for readability within the source code.
What we need is not an entity, but a way to indicate semantically that the rendered content contains a line-break at this point. We also need to not indicate anything else (we can already indicate a line-break by beginning or ending a block element, but that's not what we want). The only reasonable way to do so is to have an element that means exactly that, and so we have the <br/> element, with its related tag being put into the source code.
A tag and a character entity reference exist for different reasons - character entities are stand-ins for certain characters (sometimes required as escape sequences - for example & for an ampersand &), tags are there for structure.
The reason the <br> tag exists is that HTML collapses whitespace. There needs to be a way to specify a hard line break - a place that has to have a line break. This is the function of the <br> tag.
There is no single character that has this meaning, though U+2028 LINE SEPARATOR has similar meaning, and even if it were to be used it would not help as it is considered to be whitespace and HTML would collapse it.
See the answers from #John Kugelman and #John Hanna for more detail on this aspect.
Not entirely related, there is another reason why a &br; character entity reference does not exist: a line break is defined in such a way that it could have more than one character, see the HTML 4 spec:
A line break is defined to be a carriage return (
), a line feed (
), or a carriage return/line feed pair.
Character entities are single character escapes, so cannot represent this, again in the HTML 4 spec:
A character entity reference is an SGML construct that references a character of the document character set.
You will see that all the defined character entities map to a single character. A line break/new line cannot be cleanly mapped this way, thus an entity is required instead of a character entity reference.
This is why a line break cannot be represented by a character entity reference.
Regardless, it not not needed as simply using the Enter key inserts a line break.
Entities are stand-ins for other characters or bits of text. In HTML they are used to represent characters that are hard to type (e.g. — for "—") or for characters that need to be escaped (& for "&"). What would a hypothetical &br; entity stand for?
It couldn't be \r or \n or \r\n as these are already easy enough to type (just press enter). The issue you're trying to workaround is that HTML collapses whitespace in most contexts and treats newlines as spaces. That is, \n is not a line break character, it is just whitespace like tabs and spaces.
An entity &br; would have to be replaced by some other text. What character do you use to represent the concept of "hard line break"? The standard line break character \n is exactly the right character, but unfortunately it's unsuitable since it's thrown in the generic "whitespace" bucket. You'd have to either overload some other control character to represent "hard line break", or use some extended Unicode character. When HTML was designed Unicode was only a nascent, still-developing standard, so that wasn't an option.
A <br> element was the simple, straightforward way to add the concept of "hard line break" to a document since no character could represent that concept.
In HTML all line breaks are treated as white space:
A line break is defined to be a carriage return (
), a line feed (
), or a carriage return/line feed pair. All line breaks constitute white space.
And white space does only separate words and sequences of white space is collapsed:
For all HTML elements except PRE, sequences of white space separate "words" (we use the term "word" here to mean "sequences of non-white space characters"). […]
[…]
Note that a sequence of white spaces between words in the source document may result in an entirely different rendered inter-word spacing (except in the case of the PRE element). In particular, user agents should collapse input white space sequences when producing output inter-word space. […]
This means that line breaks cannot be expressed by plain characters. And although there are certain special characters in Unicode to unambiguously separate lines and paragraphs, they are not specified to do this in HTML too:
Note that although
and
are defined in [ISO10646] to unambiguously separate lines and paragraphs, respectively, these do not constitute line breaks in HTML […]
That means there is no plain character or sequence of plain characters that is to mark a line break in HTML. And that’s why there is the BR element.
Now if you want to use &br; instead of <br>, you just need to declare the entity br to represent the value <br>:
<!ENTITY br "<br>">
Having this additional entity named br declared, a general-purpose XML or SGML processor will replace every occurrence of the entity reference &br; with the value it represents (<br>). An example document:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd" [
<!ENTITY br "<br>">
]>
<HTML>
<HEAD>
<TITLE>My first HTML document</TITLE>
</HEAD>
<BODY>
<P>Hello &br;world!
</BODY>
</HTML>
Entities are content, tags are structure or layout (very roughly speaking). It seems whoever made the <br> a tag decided that breaking a line has more to do with structure and layout than with content. Not being able to actually "see" a <br>
I'd tend to agree. Oh and I'm making this up as I go so feel free to disagree ;)
HTML is a mark-up language - it represents the structure of a document, not how that document should appear visually. Take the <EM> tag as an example - it tells user-agents that they should give emphasis to any text that is placed between the opening and closing <EM> tags. However, it does not state how that emphasis should be represented. Yes, most visual web-browsers will place the text in italics, but this is only convention. Other browsers, such as monochrome text-only browsers may display the text in inverse. A screen reader might read the text in a louder voice, or change the pronunciation. A search-engine spider might decide the text is more important than other elements.
The same goes for the <BR> tag - it isn't just another character entity, it actually represents a break in the document structure. A <BR> is not just a replacement for a newline character, but is a "semantic" part of the document and how it is structured. This is similar to the way an <H1> is not just a way of making text bigger and bolder, but is an integral part of the way the document is structured.
br elements can be styled, though. How would you style an HTML entity? Because they're elements it makes them more flexible.
Yes. An HTML entity would be more appropriate, as a break tag cannot contain text and behaves much like a newline.
That's just not the way things are, though. Too late. I can't tell you the number of non-XML-compatible HTML documents I've had to deal with because of unclosed break tags...