Parse broken HTML with golang

Question

I need to find elements in an HTML string. Unfortunately the HTML is pretty much broken (e.g. closing tags without an opening pair).

I tried to use XPath with launchpad.net/xmlpath but it can't parse an HTML file so damn buggy.

How can I find elements in a broken HTML with golang? I would prefer using XPath, but I am open for other solutions too if I can use it to look for tags with a specific id or class.

For those stumbling on this issue now, note that the xmlpath project has moved (and improved) to gopkg.in/xmlpath.v1 . — ChrisR
– ChrisR, Commented Apr 16, 2015 at 20:50

Jeremy Quinton · Accepted Answer · 2016-06-01 12:10:24Z

21

It seems net/html does the job.

So that's what I am doing now:

package main

import (
    "strings"
    "golang.org/x/net/html"
    "log"
    "bytes"
    "gopkg.in/xmlpath.v2"
)

func main() {
    brokenHtml := `<!DOCTYPE html><html><body><h1 id="someid">My First Heading</h1><p>paragraph</body></html>`

    reader := strings.NewReader(brokenHtml)
    root, err := html.Parse(reader)

    if err != nil {
        log.Fatal(err)
    }

    var b bytes.Buffer
    html.Render(&b, root)
    fixedHtml := b.String()

    reader = strings.NewReader(fixedHtml)
    xmlroot, xmlerr := xmlpath.ParseHTML(reader)

    if xmlerr != nil {
        log.Fatal(xmlerr)
    }

    var xpath string
    xpath = `//h1[@id='someid']`
    path := xmlpath.MustCompile(xpath)
    if value, ok := path.String(xmlroot); ok {
        log.Println("Found:", value)
    }
}

edited Jun 1, 2016 at 12:10

Jeremy Quinton

6864 silver badges13 bronze badges

answered Jun 8, 2014 at 1:12

mimrock

4,8939 gold badges36 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ChrisR Over a year ago

Do you know how to iterate over all Nodes which match a given XPath? Thanks.

Kreisquadratur Over a year ago

iter := path.Iter(xmlroot) for iter.Next() { log.Println(iter.Node().String()) }

Collectives™ on Stack Overflow

Parse broken HTML with golang

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related