I have a bunch of records in a QuickBase table that contain a rich text field. In other words, they each contain some paragraphs of text intermingled with HTML tags like <p>, <strong>, etc.
I need to migrate the records to a new table where the corresponding field is a plain text field. For this, I would like to strip out all HTML tags and leave only the text in the field values.
For example, from the below input, I would expect to extract just a small example link to a webpage:
<p>just a small <a href="#">
example</a> link</p><p>to a webpage</p>
As I am trying to get this done quickly and without coding or using an external tool, I am constrained to using Quickbase Pipelines' Text channel tool. The way it works is that I define a regex pattern and it outputs only the bits that match the pattern.
So far I've been able to come up with this regular expression (Python-flavored as QB's backend is written in Python) that correctly does the exact opposite of what I need. I.e. it matches only the HTML tags:
/(<[^>]*>)/
In a sense, I need the negative image of this expression but have not be able to build it myself.
Your help in "negating" the above expression is most appreciated.
Assuming there are no < or > elsewhere or entity-encoded, an idea using a lookbehind.
(?:(?<=>)|^)[^<]+
See this demo at regex101
(?:(?<=>)|^) is an alternation between either ^ start of the string or looking behind for any >. From there [^<]+ matches one or more characters that are not < (negated character class).
So recently I found a question that was
Which of the following is a valid tag?
<213person>
<_person> (This is given as the right answer)
Both
None
(Note: this is the explanation that was given:- Valid HTML tags are surrounded by the angle brackets and the tag name can only either start from an alphabet or an underscore(_))
As far as my knowledge goes none of the reserved tags start with an underscore and according to what I've read about custom HTML tags it has to start with an alphabet(I tested it and it doesn't work with a custom tag starting with any character that's not an alphabet). So in my opinion and according to what I tested HTML tags can only start with alphabets or! (in case of !-- -- and !DOCTYPE HTML)
What I want to know is if the given explanation is correct or not and if it's correct then can someone provide some proper documentation and working examples for it?
As mentioned by #Rob, the standard defines a valid tag name as string containing alphanumeric ASCII characters, being:
0-9|a-z|A-Z
However, browsers handle things differently.
There's a few main points that I've noticed which don't align with the current standard.
Tag names must start with a letter
If a tag name starts with any character outside a-z|A-Z, the start tag ends up being interpreted as text and the end tag gets converted into a comment.
Special characters can be used
The following HTML is valid in a lot of browsers and will create an element:
<Z[\]^_`a></Z[\]^_`a>
This seems to be browsers only checking if the characters are ASCII. The only exception is the first character (as stated above).
Initially, I thought this was a simplified check, so instead of [A-Z]|[a-z| they checked [A-z], but you can use any character outside this range.
This makes the following HTML also "valid" in the eyes of certain browsers:
<a!></a!>
<aʬ></aʬ>
<a͢͢͢></a͢͢͢>
<a͢͢͢ʬ͢ʬ͢ʬ͢ʬ͢ʬ͢ʬ͢ΘΘΘΘ></a͢͢͢ʬ͢ʬ͢ʬ͢ʬ͢ʬ͢ʬ͢ΘΘΘΘ>
<a></a>
I tested the HTML elements in both Chrome and Firefox, I didn't test any other browsers. I also didn't test every ASCII character, just some very high and low in terms of their character code.
From the HTML standard:
Start tags must have the following format:
The first character of a start tag must be a U+003C LESS-THAN SIGN
character (<). The next few characters of a start tag must be the
element's tag name.
So what is allowed in the element's tag name? This is defined just above:
Tags contain a tag name, giving the element's name. HTML elements all
have names that only use ASCII alphanumerics. In the HTML syntax, tag
names, even those for foreign elements, may be written with any mix of
lower- and uppercase letters that, when converted to all-lowercase,
matches the element's tag name; tag names are case-insensitive.
On a webpage I found a tag that begins with a Unicode letter 休
Is there a list somewhere of the letters and symbols may validly follow right after the less than sign?
They're not using a mark-up less than sign "<" on their site but rather they are using the HTML entity less than < to display the reserved character as text rather than HTML.
This can be treated just like ordinary text. So in essence, it's not a tag, its just ordinary text.
For instance the line:
<font style="color:#F00;"><休闲文化></font>
Actually is:
<font style="color:#F00;"><休闲文化></font>
Thus, <休闲文化> isn't a tag itself, but rather just text (which uses HTML reserved characters within it - perhaps marking you confuse it for a tag)
In which context is it used? XML, HTML,...?
In case of HTML there are tags already defined, you can't use a random one. In XML you can define you're own tags. In both cases you might use random tags, while not ending up with error you would notice, the tag might just get skipped.
I believe this Wiki page might help you:
https://en.m.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references
All characters you want (but you should quote few).
As you notice, the character <休 come in <a href="http:(...)" target="_blank" title="<休闲文化>【成都大熊猫基. So this is inside a string (an attribute value). The < in this case will not indicate a start of a tag.
I want to make newline (CR and LF) characters visible in a textarea field of an HTML form, as you can do in some text editors and IDEs. The user needs to be able to edit the text to insert newlines as well (i.e. create paragraph breaks), which should also show dynamically. Is there a way to do this?
TIA....
Steve
The only way to do this is to print out your own marker characters before/after each line break, using javascript.
The character reference for the pilcrow (¶) character is ¶.
Why indeed? Wouldn't something like &br; be more appropriate?
An HTML entity reference is, depending on HTML version either an SGML entity or an XML entity (HTML inherits entities from the underlying technology). Entities are a way of inserting chunks of content defined elsewhere into the document.
All HTML entities are single-character entities, and are hence basically the same as character references (technically they are different to character references, but as there are no multi-character entities defined, the distinction has no impact on HTML).
When an HTML processor sees, for example — it replaces it with the content of that entity reference with the appropriate entity, based on the section in the DTD that says:
<!ENTITY mdash CDATA "—" -- em dash, U+2014 ISOpub -->
So it replaces the entity reference with the entity — which is in turn a character reference that gets replaced by the character — (U+2014). In reality unless you are doing this with a general-purpose XML or SGML processor that doesn't understand HTML directly, this will really be done in one step.
Now, what would we replace your hypothetical &br; with to cause a line-break to happen? We can't do so with a newline character, or even the lesser known U+2028 LINE SEPARATOR (which semantically in plain text has the same meaning as <br/> in HTML), because they are whitespace characters which are not significant in most HTML code, which is something that you should be grateful for as writing HTML would be much harder if we couldn't format for readability within the source code.
What we need is not an entity, but a way to indicate semantically that the rendered content contains a line-break at this point. We also need to not indicate anything else (we can already indicate a line-break by beginning or ending a block element, but that's not what we want). The only reasonable way to do so is to have an element that means exactly that, and so we have the <br/> element, with its related tag being put into the source code.
A tag and a character entity reference exist for different reasons - character entities are stand-ins for certain characters (sometimes required as escape sequences - for example & for an ampersand &), tags are there for structure.
The reason the <br> tag exists is that HTML collapses whitespace. There needs to be a way to specify a hard line break - a place that has to have a line break. This is the function of the <br> tag.
There is no single character that has this meaning, though U+2028 LINE SEPARATOR has similar meaning, and even if it were to be used it would not help as it is considered to be whitespace and HTML would collapse it.
See the answers from #John Kugelman and #John Hanna for more detail on this aspect.
Not entirely related, there is another reason why a &br; character entity reference does not exist: a line break is defined in such a way that it could have more than one character, see the HTML 4 spec:
A line break is defined to be a carriage return (
), a line feed (
), or a carriage return/line feed pair.
Character entities are single character escapes, so cannot represent this, again in the HTML 4 spec:
A character entity reference is an SGML construct that references a character of the document character set.
You will see that all the defined character entities map to a single character. A line break/new line cannot be cleanly mapped this way, thus an entity is required instead of a character entity reference.
This is why a line break cannot be represented by a character entity reference.
Regardless, it not not needed as simply using the Enter key inserts a line break.
Entities are stand-ins for other characters or bits of text. In HTML they are used to represent characters that are hard to type (e.g. — for "—") or for characters that need to be escaped (& for "&"). What would a hypothetical &br; entity stand for?
It couldn't be \r or \n or \r\n as these are already easy enough to type (just press enter). The issue you're trying to workaround is that HTML collapses whitespace in most contexts and treats newlines as spaces. That is, \n is not a line break character, it is just whitespace like tabs and spaces.
An entity &br; would have to be replaced by some other text. What character do you use to represent the concept of "hard line break"? The standard line break character \n is exactly the right character, but unfortunately it's unsuitable since it's thrown in the generic "whitespace" bucket. You'd have to either overload some other control character to represent "hard line break", or use some extended Unicode character. When HTML was designed Unicode was only a nascent, still-developing standard, so that wasn't an option.
A <br> element was the simple, straightforward way to add the concept of "hard line break" to a document since no character could represent that concept.
In HTML all line breaks are treated as white space:
A line break is defined to be a carriage return (
), a line feed (
), or a carriage return/line feed pair. All line breaks constitute white space.
And white space does only separate words and sequences of white space is collapsed:
For all HTML elements except PRE, sequences of white space separate "words" (we use the term "word" here to mean "sequences of non-white space characters"). […]
[…]
Note that a sequence of white spaces between words in the source document may result in an entirely different rendered inter-word spacing (except in the case of the PRE element). In particular, user agents should collapse input white space sequences when producing output inter-word space. […]
This means that line breaks cannot be expressed by plain characters. And although there are certain special characters in Unicode to unambiguously separate lines and paragraphs, they are not specified to do this in HTML too:
Note that although
and
are defined in [ISO10646] to unambiguously separate lines and paragraphs, respectively, these do not constitute line breaks in HTML […]
That means there is no plain character or sequence of plain characters that is to mark a line break in HTML. And that’s why there is the BR element.
Now if you want to use &br; instead of <br>, you just need to declare the entity br to represent the value <br>:
<!ENTITY br "<br>">
Having this additional entity named br declared, a general-purpose XML or SGML processor will replace every occurrence of the entity reference &br; with the value it represents (<br>). An example document:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd" [
<!ENTITY br "<br>">
]>
<HTML>
<HEAD>
<TITLE>My first HTML document</TITLE>
</HEAD>
<BODY>
<P>Hello &br;world!
</BODY>
</HTML>
Entities are content, tags are structure or layout (very roughly speaking). It seems whoever made the <br> a tag decided that breaking a line has more to do with structure and layout than with content. Not being able to actually "see" a <br>
I'd tend to agree. Oh and I'm making this up as I go so feel free to disagree ;)
HTML is a mark-up language - it represents the structure of a document, not how that document should appear visually. Take the <EM> tag as an example - it tells user-agents that they should give emphasis to any text that is placed between the opening and closing <EM> tags. However, it does not state how that emphasis should be represented. Yes, most visual web-browsers will place the text in italics, but this is only convention. Other browsers, such as monochrome text-only browsers may display the text in inverse. A screen reader might read the text in a louder voice, or change the pronunciation. A search-engine spider might decide the text is more important than other elements.
The same goes for the <BR> tag - it isn't just another character entity, it actually represents a break in the document structure. A <BR> is not just a replacement for a newline character, but is a "semantic" part of the document and how it is structured. This is similar to the way an <H1> is not just a way of making text bigger and bolder, but is an integral part of the way the document is structured.
br elements can be styled, though. How would you style an HTML entity? Because they're elements it makes them more flexible.
Yes. An HTML entity would be more appropriate, as a break tag cannot contain text and behaves much like a newline.
That's just not the way things are, though. Too late. I can't tell you the number of non-XML-compatible HTML documents I've had to deal with because of unclosed break tags...