Auto generate XPath for known element in HTML tree using python

Question

Is there any way (libs, not manually) for generating relative XPath for a known element in HTML?

Let say the second P element inside class="content"

<html>
    <body>
        <div class"title">
            <h1>***</h1>
        </div>
        <p> *** </p>
        <h3>***</h3>
        <div class"content">
            <p>****</p>
            <p>****</p>
        </div>
    </body>
</html>

Use case: The idea is to guess where are the elements that I might be interested in. For example title, content or author. After I've found the element I want to generate xpath for it

I wrote couple of chrome extensions to generate the xpaths for the elements but this seems to be a special requirement where you want to get the xpath based on other element. I am sure it's going to be little tricky but it's possible. — supputuri
– supputuri, Commented Mar 28, 2020 at 22:07

Jack Fleeting · Accepted Answer · 2020-03-28 23:10:52Z

1

Try something like this:

from lxml import etree

datum = """
<html>
    <body>
        <div class="title">
            <h1>***</h1>
        </div>
        <p> *** </p>
        <h3>***</h3>
        <div class="content">
            <p>something</p>
            <p>target</p>
        </div>
    </body>
</html>
"""

root = etree.fromstring(datum)
tree = etree.ElementTree(root)
find_text = etree.XPath("//p[text()='target']")
for target in find_text(root):
    print(tree.getpath(target))

Output:

/html/body/div[2]/p[2]

answered Mar 28, 2020 at 23:10

Jack Fleeting

25k6 gold badges27 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

octopus Over a year ago

Lxml was my first option, As you can see in the output the XPath is absolute, not relative it seems I have to build it my self something like this Auto-generate XPaths using Python

Collectives™ on Stack Overflow

Auto generate XPath for known element in HTML tree using python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related