Golang xml unmarshal html table - html

Question

I have a simple HTML table, and want to get all cell values even if it's HTML code inside.
Trying to use xml unmarshal, but didn't get the right struct tags, values or attributes.
import (
"fmt"
"encoding/xml"
)
type XMLTable struct {
XMLName xml.Name `xml:"TABLE"`
Row []struct{
Cell string `xml:"TD"`
}`xml:"TR"`
}
func main() {
raw_html_table := `
<TABLE><TR>
<TD>lalalal</TD>
<TD>papapap</TD>
<TD>fafafa</TD>
<TD>
<form action=\"/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf" method=POST>
<input type=hidden name=acT value=\"Dev\">
<input type=hidden name=acA value=\"Anyval\">
<input type=submit name=submit value=Stop>
</form>
</TD>
</TR>
</TABLE>`
table := XMLTable{}
fmt.Printf("%q\n", []byte(raw_html_table)[:15])
err := xml.Unmarshal([]byte(raw_html_table), &table)
if err != nil {
fmt.Printf("error: %v", err)
}
}
As an additional info, I don't care about cell content if it's HTML code (take only []byte / string values). So I may delete cell content before unmarshaling, but this way is also not so easy.
Any suggestions with standard golang libs would be welcome.

Answer 1

Sticking to the standard lib
Your input is not valid XML, so even if you model it right, you won't be able to parse it.
First, you're using a raw string literal to define your input HTML as a string, and raw string literals cannot contain escapes. For example this:
<form action=\"/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf" method=POST>
You can't use \" in a raw string literal (you can, but it will mean exactly those 2 characters), and you don't have to, use a simple quotation mark: ".
Next, in XML you cannot have attributes without putting their values in quotes.
Third, each element must have a matching closing element, your <input> elements are not closed.
So for example this line:
<input type=hidden name=acT value=\"Dev\">
Must be changed to:
<input type="hidden" name="acT" value="Dev" />
Ok, after these the input is a valid XML now.
How to model it? Simple as this:
type XMLTable struct {
Rows []struct {
Cell string `xml:",innerxml"`
} `xml:"TR>TD"`
}
And the full code to parse and print contents of <TD> elements:
raw_html_table := `
<TABLE><TR>
<TD>lalalal</TD>
<TD>papapap</TD>
<TD>fafafa</TD>
<TD>
<form action="/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf" method="POST">
<input type="hidden" name="acT" value="Dev" />
<input type="hidden" name="acA" value="Anyval" />
<input type="submit" name="submit" value="Stop" />
</form>
</TD>
</TR>
</TABLE>`
table := XMLTable{}
err := xml.Unmarshal([]byte(raw_html_table), &table)
if err != nil {
fmt.Printf("error: %v\n", err)
}
fmt.Println("count:", len(table.Rows))
for _, row := range table.Rows {
fmt.Println("TD content:", row.Cell)
}
Output (try it on the Go Playground):
count: 4
TD content: lalalal
TD content: papapap
TD content: fafafa
TD content:
<form action="/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf" method="POST">
<input type="hidden" name="acT" value="Dev" />
<input type="hidden" name="acA" value="Anyval" />
<input type="submit" name="submit" value="Stop" />
</form>
Using a proper HTML parser
If you can't or don't want to change the input HTML, or you want to handle all HTML input not just valid XMLs, you should use a proper HTML parser instead of treating the input as XML.
Check out https://godoc.org/golang.org/x/net/html for an HTML5-compliant tokenizer and parser.

Answer 2

Once your input is valid HTML (your snippet is missing quotes in attributes), you can configure a xml.Decoder with entities and autoclose maps (and make it non-strict), which will end up working :
Run my modified version here.
package main
import (
"encoding/xml"
"fmt"
"strings"
)
type XMLTable struct {
Rows []struct {
Cell string `xml:",innerxml"`
} `xml:"TR>TD"`
}
func main() {
raw_html_table := `
<TABLE><TR>
<TD>lalalal</TD>
<TD>papapap</TD>
<TD>fafafa</TD>
<TD>
<form action="/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf" method="POST">
<input type="hidden" name="acT" value="Dev">
<input type="hidden" name="acA" value="Anyval">
<input type="submit" name="submit" value="Stop">
</form>
</TD>
</TR>
</TABLE>`
table := XMLTable{}
decoder := xml.NewDecoder(strings.NewReader(raw_html_table))
decoder.Entity = xml.HTMLEntity
decoder.AutoClose = xml.HTMLAutoClose
decoder.Strict = false
err := decoder.Decode(&table)
if err != nil {
fmt.Printf("error: %v", err)
}
fmt.Printf("%#v\n", table)
}