How long is too long for an attribute value in HTML?
I'm using HTML5 style data attributes (data-foo="bar") in a new application, and in one place it would be really handy to store a fair whack of data (upwards of 100 characters). While I suspect that this amount is fine, it raises the question of how much is too much?
HTML5 has no limits on the length of attribute values.
As the spec says, "This version of HTML thus returns to a non-SGML basis."
Later on, when describing how to parse HTML5, the following passage appears (emphasis added):
The algorithm described below places
no limit on the depth of the DOM tree
generated, or on the length of tag
names, attribute names, attribute
values, text nodes, etc. While
implementors are encouraged to avoid
arbitrary limits, it is recognized
that practical concerns will likely
force user agents to impose nesting
depth constraints.
Therefore, (theoretically) there is no limit to the length/size of HTML5 attributes.
See revision history for original answer covering HTML4.
I've just written a test (Note! see update below) which puts a string of length 10 million into an attribute and then retrieves it again, and it works fine (Firefox 3.5.2 & Internet Explorer 7)
50 million makes the browser hang with the "This script is taking a long time to complete" message.
Update: I've fixed the script: it previously set the innerHTML to a long string and now it's setting a data attribute. https://output.jsbin.com/wikulamuni It works for me with a length of 100 million. YMMV.
el.setAttribute('data-test', <<a really long string>>)
I really don't think there is any limit. I know now you can do
<a onclick=" //...insert 100KB of javascript code here">
and it works fine. Albeit a little unreadable.
From HTML5 syntax doc
9.1.2.3 Attributes
Attributes for an element are
expressed inside the element's start
tag.
Attributes have a name and a value.
Attribute names must consist of one or
more characters other than the space
characters, U+0000 NULL, U+0022
QUOTATION MARK ("), U+0027 APOSTROPHE
('), U+003E GREATER-THAN SIGN (>),
U+002F SOLIDUS (/), and U+003D EQUALS
SIGN (=) characters, the control
characters, and any characters that
are not defined by Unicode. In the
HTML syntax, attribute names may be
written with any mix of lower- and
uppercase letters that are an ASCII
case-insensitive match for the
attribute's name.
Attribute values are a mixture of text
and character references, except with
the additional restriction that the
text cannot contain an ambiguous
ampersand.
Attributes can be specified in four
different ways:
Empty attribute syntax
Unquoted attribute value syntax
Single-quoted attribute value syntax
Double-quoted attribute value syntax
Here there hasn't mentioned a limit on the size of the attribute value. So I think there should be none.
You can also validate your document against the
HTML5 Validator(Highly Experimental)
I've never heard of any limit on the length of attributes.
In the HTML 4.01 specifications, in the section on Attributes there is nothing that mention any limitation on this.
Same in the HTML 4.01 DTD -- in fact, as far as I know, DTD don't allow you to specify a length to attributes.
If there is nothing about that in HTML 4, I don't imagine anything like that would appear for HTML 5 -- and I actually don't see any length limitation in the 9.1.2.3 Attributes section for HTML 5 either.
Tested recently in Edge (Version 81.0.416.58 (64 bits)), and data-attributes seem to have a limit of 64k.
From http://dev.w3.org/html5/spec/Overview.html#embedding-custom-non-visible-data:
Every HTML element may have any number of custom data attributes specified, with any value.
That which is used to parse/process these data-* attribute values will have
limitations.
Turns out the data-attributes and values are placed in a DOMStringMap object.
This has no inherent limits.
From http://dev.w3.org/html5/spec/Overview.html#domstringmap:
Note: The DOMStringMap interface definition here is only intended for JavaScript
environments. Other language bindings will need to define how DOMStringMap is to be
implemented for those languages
DOMStringMap is an interface with a getter, setter, greator and deleter.
The setter has two parameters of type DOMString, name and value.
The value is of type DOMString that is is mapped directly to a JavaScript String.
From https://bytes.com/topic/javascript/answers/92088-max-allowed-length-javascript-string:
The maximum length of a JavaScript String is implementation specific.
The SGML Defines attributes with a limit set of 65k characters, seen here:
http://www.highdots.com/forums/html/length-html-attribute-175546.html
Although for what you are doing, you should be fine.
As for the upper limits, I have seen jQuery use data attributes hold a few k of data personally as well.
Related
How long is too long for an attribute value in HTML?
I'm using HTML5 style data attributes (data-foo="bar") in a new application, and in one place it would be really handy to store a fair whack of data (upwards of 100 characters). While I suspect that this amount is fine, it raises the question of how much is too much?
HTML5 has no limits on the length of attribute values.
As the spec says, "This version of HTML thus returns to a non-SGML basis."
Later on, when describing how to parse HTML5, the following passage appears (emphasis added):
The algorithm described below places
no limit on the depth of the DOM tree
generated, or on the length of tag
names, attribute names, attribute
values, text nodes, etc. While
implementors are encouraged to avoid
arbitrary limits, it is recognized
that practical concerns will likely
force user agents to impose nesting
depth constraints.
Therefore, (theoretically) there is no limit to the length/size of HTML5 attributes.
See revision history for original answer covering HTML4.
I've just written a test (Note! see update below) which puts a string of length 10 million into an attribute and then retrieves it again, and it works fine (Firefox 3.5.2 & Internet Explorer 7)
50 million makes the browser hang with the "This script is taking a long time to complete" message.
Update: I've fixed the script: it previously set the innerHTML to a long string and now it's setting a data attribute. https://output.jsbin.com/wikulamuni It works for me with a length of 100 million. YMMV.
el.setAttribute('data-test', <<a really long string>>)
I really don't think there is any limit. I know now you can do
<a onclick=" //...insert 100KB of javascript code here">
and it works fine. Albeit a little unreadable.
From HTML5 syntax doc
9.1.2.3 Attributes
Attributes for an element are
expressed inside the element's start
tag.
Attributes have a name and a value.
Attribute names must consist of one or
more characters other than the space
characters, U+0000 NULL, U+0022
QUOTATION MARK ("), U+0027 APOSTROPHE
('), U+003E GREATER-THAN SIGN (>),
U+002F SOLIDUS (/), and U+003D EQUALS
SIGN (=) characters, the control
characters, and any characters that
are not defined by Unicode. In the
HTML syntax, attribute names may be
written with any mix of lower- and
uppercase letters that are an ASCII
case-insensitive match for the
attribute's name.
Attribute values are a mixture of text
and character references, except with
the additional restriction that the
text cannot contain an ambiguous
ampersand.
Attributes can be specified in four
different ways:
Empty attribute syntax
Unquoted attribute value syntax
Single-quoted attribute value syntax
Double-quoted attribute value syntax
Here there hasn't mentioned a limit on the size of the attribute value. So I think there should be none.
You can also validate your document against the
HTML5 Validator(Highly Experimental)
I've never heard of any limit on the length of attributes.
In the HTML 4.01 specifications, in the section on Attributes there is nothing that mention any limitation on this.
Same in the HTML 4.01 DTD -- in fact, as far as I know, DTD don't allow you to specify a length to attributes.
If there is nothing about that in HTML 4, I don't imagine anything like that would appear for HTML 5 -- and I actually don't see any length limitation in the 9.1.2.3 Attributes section for HTML 5 either.
Tested recently in Edge (Version 81.0.416.58 (64 bits)), and data-attributes seem to have a limit of 64k.
From http://dev.w3.org/html5/spec/Overview.html#embedding-custom-non-visible-data:
Every HTML element may have any number of custom data attributes specified, with any value.
That which is used to parse/process these data-* attribute values will have
limitations.
Turns out the data-attributes and values are placed in a DOMStringMap object.
This has no inherent limits.
From http://dev.w3.org/html5/spec/Overview.html#domstringmap:
Note: The DOMStringMap interface definition here is only intended for JavaScript
environments. Other language bindings will need to define how DOMStringMap is to be
implemented for those languages
DOMStringMap is an interface with a getter, setter, greator and deleter.
The setter has two parameters of type DOMString, name and value.
The value is of type DOMString that is is mapped directly to a JavaScript String.
From https://bytes.com/topic/javascript/answers/92088-max-allowed-length-javascript-string:
The maximum length of a JavaScript String is implementation specific.
The SGML Defines attributes with a limit set of 65k characters, seen here:
http://www.highdots.com/forums/html/length-html-attribute-175546.html
Although for what you are doing, you should be fine.
As for the upper limits, I have seen jQuery use data attributes hold a few k of data personally as well.
I am writing a markup document in Finnish.
I'm using the lang="fi-fi" attribute. Am I supposed to use the markup entities (ä for ä etc.) in conjunction with the language attribute, or is using the language attribute alone sufficient? How do the entities and language attribute affect each other?
The "problem" comes from the fact that the markup is written without entities and I have a script that's supposed to replace the scandic letters with entities by using regular expressions -- after defining the lang attribute the script doesn't appear to work anymore (which it supposedly did before adding the lang attribute).
My main concern is that the markup renders correctly regardless of the browser, although a "modern" browser can be assumed.
The lang attribute and entities do completely different jobs.
The lang attribute tells the parser what human language the document is written in. This allows, for example, search engines to tell if it is a good document to present to Finish speakers and screen reader software to select the correct pronunciation library.
Entities just let you represent characters that you couldn't otherwise represent. e.g.
Because you can't type the character of your keyboard
Because the character encoding the document is saved in (e.g. ASCII) doesn't include the character. This century you should be using UTF-8 just about everywhere and shouldn't need to worry about that.
Because the character would otherwise have special meaning in HTML (e.g. <).
Always use a lang attribute if you know what language the text of the document will be written in
Always use entities for characters with special meaning in HTML
Use literal characters if you can be reasonably certain the character encoding won't be mangled (which you can be most of the time) as they use fewer bytes and are easier to read in source code.
The root of my problem was actually character encoding. Although all of the documents were defined with UTF-8, the script somehow didn't recognize it. By telling the script that the input files (that were supposed be fixed with entities) are UTF-8 encoded the script functions correctly again.
As an answer to the question in the heading: to be absolutely sure that the documents are compatible with the server -- yes, I am supposed to use entity encoding (though I understand that assuming that the server allows UTF-8 is pretty safe assumption in general as implied by Quentin). Due to other reasons (related to automatic content generating), I'm also supposed to use the lang attribute.
I'd like to remove more unnecessary bytes from my output, and it seems it's acceptable (in practice) to strip what can add up to quite a lot of whitespace from HTML markup by omitting/collapsing the gaps between DOM element attributes.
Although I've tested and researched (a little in both cases), I'm wondering how safe it would be?
I tested in Chrome (43.0.2357.65 m), IE (11.0.9600.17801), FF (38.0.1) and Safari (5.1.7 (blah-di-blah)) and they didn't seem to mind, and couldn't find anything specific in The Specs about whitespace between attributes.
w3.org's Validator complains, which is a strong indication that this is not safe and shouldn't be expected to work, but (there's always a "but") it's possible the requirement for a space is only strict when no quotes are present (for obvious reasons).
Also (snippy but poignant): their SSL is "out of date" which doesn't inspire confidence in their opinion.
I noted also that someone's HTML compressor could (when enabled) strip quotes around attribute values where those values had no whitespace within them (e.g. id), which implies that at least most if not all HTML parsing is focussed on the text either side of the equals signs (except with booleans of course), and where quotes are in use, they'd be considered the prioritized delimiter.
So, would:
<!DOCTYPE html><html><body>
Yabba Dabba Doo!
</body></html>
▲ that ever go wrong, and if so, under which conditions?
What other reasons could there be to maintain this whitespace in production output (code "readability" is a non issue in this case)?
Update (since finding an answer):
Although I basically answered my own question insofar that there is a specification governing whether there should be a space between attributes, I still wonder if omitting them when using quoted values can be considered practically safe, and would appreciate feedback on this point.
Considering how often spaces may be omitted by accident in production HTML, and that the browsers I tested don't seem to mind when they are, I assume it would be very rare if ever that a browser failed to handle documents with these spaces omitted.
Although it's sensible to follow the specs in pretty much all situations, might this be one time cheating a bit could be acceptable?
After all - if we can magically save several hundred bytes without affecting the quality of the output, why not?
There is a specification (after all)
It turns out I should have looked harder. My bad.
According to these specs:
If an attribute using the empty attribute syntax is to be followed by another attribute, then there must be a space character separating the two.
and
If an attribute using the unquoted attribute syntax is to be followed by another attribute or by the optional U+002F SOLIDUS character (/) allowed in step 6 of the start tag syntax above, then there must be a space character separating the two.
and
If an attribute using the single-quoted attribute syntax is to be followed by another attribute, then there must be a space character separating the two.
and
If an attribute using the double-quoted attribute syntax is to be followed by another attribute, then there must be a space character separating the two.
Which unless I am mistaken (again), means there must always be spaces between attributes.
You could try online HTML minifiers like http://www.whak.ca/minify/HTML.htm or http://www.scriptcompress.com/minify-HTML.htm (search google for more) and find little things they change for hints to what can be taken out yet still render the HTML code.
On the first link your code:
<!DOCTYPE html><html><body>
Yabba Dabba Doo!
</body></html>
Turns into:
<!DOCTYPE html><html><body>Yabba Dabba Doo!
saving you 18 bytes already...
In HTML 5 specification the parser and the specification state that the element name can be everything starting with a letter and followed by alpha-numeric characters.
Now the question is what happens if I introduce additional elements not part of the specification but valid in terms of compliance to the specified syntax.
What do all those browsers do when they encounter elements with custom yet unknown name? Does those elements got treaten like any element or are they left out, stripped out or replaced?
How do for instance do HTML5 editor behave?
Is there anything in the specifications I have overlooked regarding valid element tag names?
[Update]
The specification was missleading here since it states the alpha-numerical character of the HTML element names. While reading a HTML 5 Specification, I missconcluded that this is true for all element names.
That is appearently wrong. In the parser section it states that a element name must only start with an ASCII letter and after that letter everthing except:
"tab" (U+0009)
"LF" (U+000A)
"FF" (U+000C)
U+0020 SPACE
"/" (U+002F)
">" (U+003E)
U+0000 NULL
EOF
Beside those mentioned characters which require special treatment involving errors or ending the tag name all other possible characters seam to be allowed.
Anything else
--> Append the current input character to the current tag token's tag name.
From my field test also additional uni-code letters are allowed for the first letter by several parsers (at least they are graceful with those).
[/Update]
In HTML 5 specification the parser and the specification state that
the element name can be everything starting with a letter and followed
by alpha-numeric characters.
Incorrect. The specification states that the element name must be one of the names explicitly listed in that document, or in another applicable specification. These include but are not limited to SVG and MathML.
The specification also includes a processing specification for consumers of HTML, such as browsers. This doesn't describe what's "allowed", it describes what those consumers should do with each character of the document regardless of whether it contains things that are allowed or not allowed.
Now the question is what happens if I introduce additional elements
not part of the specification but valid in terms of compliance to the
specified syntax.
The above rules are followed. The "specified syntax" is irrelevant. The specification describes what the consumer should do for any input stream of characters.
What do all those browsers do when they encounter elements with custom
yet unknown name? Does those elements got treated like any element or
are they left out, stripped out or replaced?
They are treated as elements in the http://www.w3.org/1999/xhtml namespace which implement the HTMLUnknownElement interface.
How for instance do HTML5 editor behave?
If they are HTML5 compliant they will behave the same way when reading in the HTML.
Is there anything in the specifications I have overlooked regarding
valid element tag names?
See the first paragraph above. Also the Custom Elements spec which makes any element name starting with an ASCII letter and containing a hyphen to be considered valid. It is unclear whether that specification is currently an "HTML5 applicable specification" but if not, it will very probably be one soon.
When outputting HTML, there are several different places where text can be interpreted as control characters rather than as text literals. For example, in "regular" text (that is, outside any element markup):
<div>This is regular text</div>
As well as within the values of attributes:
<input value="this is value text">
And, I believe, within HTML comments:
<!-- This text here might be programmatically generated
and could, in theory, contain the double-hyphen character
sequence, which is verboten inside comments -->
Each of these three kinds of text has different rules for how it must be escaped in order to be treated as non-markup. So my first question is, are there any other contexts in HTML in which characters can be interpreted as markup/control characters? The above contexts clearly have different rules about what needs to be escaped.
The second question is, what are the canonical, globally-safe lists of characters (for each context) that need to be escaped to ensure that any embedded text is treated as non-markup? For example, in theory you only need to escape ' and " in attribute values, since within an attribute value only the closing-delimiter character (' or " depending on which delimiter the attribute value started with) would have control meaning. Similarly, within "regular" text only < and & have control meaning. (I realize that not all HTML parsers are identical. I'm mostly interested in what is the minimum set of characters that need escaping in order to appease a spec-conforming parser.)
Tangentially: The following text will throw errors as HTML 4.01 Strict:
foo
Specifically, it says that it doesn't know what the entity "&y" is supposed to be. If you put a space after the &, however, it validates just fine. But if you're generating this on the fly, you're probably not going to want to check whether each use of & will cause a validation error, and instead just escape all & inside attribute values.
<div>This is regular text</div>
Text content: & must be escaped. < must be escaped.
If producing a document in a non-UTF encoding, characters that do not fit inside the chosen encoding must be escaped.
In XHTML (and XML in general), the sequence ]]> must not occur in text content, so in that specific case one of the characters in that sequence must be escaped, traditionally the >. For consistency, the Canonical XML specification chooses to escape > every time in text content, which is not a bad strategy for an escaping function, though you can certainly skip it for hand-authoring.
<input value="this is value text">
Attribute values: & must be escaped. The attribute value delimiter " or ' must be escaped. If no attribute value delimiter is used (don't do that) no escape is possible.
Canonical XML always chooses " as the delimiter and therefore escapes it. The > character does not need to be escaped in attribute values and Canonical XML does not. The HTML4 spec suggested encoding > anyway for backwards compatibility, but this affects only a few truly ancient and dreadful browsers that no-one remembers now; you can ignore that.
In XHTML < must be escaped. Whilst you can get away with not escaping it in HTML4, it's not a good idea.
To include tabs, CR or LF in attribute values (without them being turned into plain spaces by the attribute value normalisation algorithm) you must encode them as character references.
For both text content and attribute values: in XHTML under XML 1.1, you must escape the Restricted Characters, which are the Delete character and C0 and C1 control codes, minus tab, CR, LF and NEL. In total, [\x01-\x08\x0B\x0C\x0E-\x1F\x7F-\x84\x86-\x9F]. The null character may not be included at all even escaped in XML 1.1. Outside XML 1.1 you can't use any of these characters at all, nor is there a good reason you'd ever want to.
<!-- This text here might be programmatically generated
and could, in theory, contain the double-hyphen character
sequence, which is verboten inside comments -->
Yes, but since there is no escaping possible inside comments, there is nothing you can do about it. If you write <!-- < -->, it literally means a comment containing “ampersand-letter l-letter t-semicolon” and will be reflected as such in the DOM or other infoset. A comment containing -- simply cannot be serialised at all.
<![CDATA[ sections and <?pis in XML also cannot use escaping. The traditional solution to serialise a CDATA section including a ]]> sequence is to split that sequence over two CDATA sections so it doesn't occur together. You can't serialise it in a single CDATA section, and you can't serialise a PI with ?> in the data.
CDATA-elements like <script> and <style> in HTML (not XHTML) may not contain the </ (ETAGO) sequence as this would end the element early and then error if not followed by the end-tag-name. Since no escaping is possible within CDATA-elements, this sequence must be avoided and worked around (eg. by turning document.write('</p>') into document.write('<\/p>');. (You see a lot of more complicated silly strategies to get around this one, like calling unescape on a JS-%-encoded string; even often '</scr'+'ipt>' which is still quite invalid.)
There is one more context in HTML and XML where different rules apply, and that's in the DTD (including the internal subset in the DOCTYPE declaration, if you have one), where the % character has Special Powers and would need to be escaped to be used literally. But as an HTML document author it is highly unlikely you would ever need to go anywhere near that whole mess.
The following text will throw errors as HTML 4.01 Strict:
foo
Yes, and it's just as much an error in Transitional.
If you put a space after the &, however, it validates just fine.
Yes, under SGML rules anything but [A-Za-z] and # doesn't start parsing as a reference. Not a good idea to rely on this though. (Of course, it's not well-formed in XHTML.)
The above contexts clearly have different rules about what needs to be escaped.
I'm not sure that the different elements have different encoding rules like you say. All the examples you list require the HTML encoding.
E.g.
<h1>Fish & Chips</h1>
<img alt="Awesome picture of Meat Pie & Chips" />
Fish & Chips
The last example includes some URL Encoding for the ampersand too (&) and its at this point things get hairy (sending an ampersand as data, which is why it must be encoded).
So my first question is, are there any other contexts in HTML in which characters can be interpreted as markup/control characters?
Anywhere within the HTML document, if the control characters are not being used as control characters, you should encode them (as a good rule of thumb). Most of the time, its HTML Encoding, & or > etc. Othertimes, when trying to pass these characters via a URL, use URL Encoding %20, %26 etc.
The second question is, what are the canonical, globally-safe lists of characters (for each context) that need to be escaped to ensure that any embedded text is treated as non-markup?
I'd say that the Wikipedia article has a few good comments on it and might be worth a read - also the W3 Schools article I guess is a good point. Most languages have built in functions to prepare text as safe HTML, so it may be worth checking your language of choice (if you are indeed even using any scripting languages and not hand coding the HTML).
Specifically, Wikipedia says: "Characters <, >, " and & are used to delimit tags, attribute values, and character references. Character entity references <, >, " and &, which are predefined in HTML, XML, and SGML, can be used instead for literal representations of the characters."
For URL Encoding, this article seems a good starting point.
Closing thoughts as I've already rambled a bit: This is all excluding the thoughts of XML / XHTML which brings a whole other ballgame to the court and its requirement that pretty much the world and its dog needs to be encoded. If you are using a scripting language and writing out a variable via that, I'm pretty sure it'll be easier to find the built in function, or download a library that'll do this for you. :) I hope this answer was scoped ok and didn't miss the point or question or come across in the wrong tone. :)
If you are looking for the best practices to escape characters in web browsers (including HTML, JavaScript and style sheets), the XSS prevention cheat sheet by Michael Coates is probably what you're looking for. It includes a description of the different interpretation contexts, tables indicating how to encode characters in each context and code samples (using ESAPI).
http://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_Prevention_Cheat_Sheet
Beware that <script> followed by <!-- followed by <script> again, enters double-escaped state, in which you probably never want to be, so ideally you should escape < with "\u003C" within your script's strings (and regexps) to not trigger it accidentally.
You can read more about it here http://qbolec-memdump.blogspot.com/2013/11/script-tag-content-madness.html
If you are this concerned about the validity of the final HTML, you might consider constructing the HTML via a DOM, versus as text.
You don't say what environment you are targeting.