Finding an href link using Python, Selenium, and XPath

Question

I want to get the href from a <p> tag using an XPath expression.

I want to use the text from <h1> tag ('Cable Stripe Knit L/S Polo') and simultaneously text from the <p> tag ('White') to find the href in the <p> tag.

Note: There are more colors of one item (more articles with different <p> tags, but the same <h1> tag)!

HTML source

<article>
    <div class="inner-article">
        <a href="/shop/tops-sweaters/ix4leuczr/a1ykz7f2b" style="height:150px;">
        </a>
        <h1>
            <a href="/shop/tops-sweaters/ix4leuczr/a1ykz7f2b" class="name-link">Cable Stripe Knit L/S Polo
            </a>
        </h1>
        <p>
            <a href="/shop/tops-sweaters/ix4leuczr/a1ykz7f2b" class="name-link">White</a>
        </p>
    </div>
</article>

I've tried this code, but it didn't work.

specificProductColor = driver.find_element_by_xpath("//div[@class='inner-article' and contains(text(), 'White') and contains(text(), 'Cable')]/p")

driver.get(specificProductColor.get_attribute("href"))

Peter Mortensen · Accepted Answer · 2022-11-10 23:58:24Z

2

As per the HTML source, the XPath expression to get the href tags would be something like this:

specificProductColors = driver.find_elements_by_xpath("//div[@class='inner-article']//a[contains(text(), 'White') or contains(text(), 'Cable')]")

specificProductColors[0].get_attribute("href")

specificProductColors[1].get_attribute("href")

Since there are two hyperlink tags, you should be using find_elements_by_xpath which returns a list of elements. In this case it would return two hyperlink tags, and you could get their href using the get_attribute method.

edited Nov 10, 2022 at 23:58

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Nov 3, 2016 at 19:45

Falloutcoder

1,0117 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Luke LubiX Long Over a year ago

Hey! Thanks for reply. Actually, what i'm trying to do is filter one product and choose specific color. There are multiple same products with different colors. (imgur.com/ei7o6rS) I tried your code, it worked but both links chose wrong color. I need the code to be as simple as possible and as fast as possible. – Luke LubiX Long 6 mins ago

Peter Mortensen · Accepted Answer · 2022-11-11 00:01:46Z

I've got working code. It's not the fastest one - this part takes approximately 550 ms, but it works. If someone could simplify that, I'd be very thankful :)

It takes all products with the specified keyword (Cable) from the product page and all products with a specified color (White) from the product page as well. It compares href links and matches wanted product with wanted color.

I also want to simplify the loop - stop both for loops if the links match.

specificProduct = driver.find_elements_by_xpath("//div[@class='inner-article']//*[contains(text(), '" + productKeyword[arrayCount] + "')]")
specificProductColor = driver.find_elements_by_xpath("//div[@class='inner-article']//*[contains(text(), '" + desiredColor[arrayCount] + "')]")

for i in specificProductColor:
    specProductColor = i.get_attribute("href")
    for i in specificProduct:
        specProduct = i.get_attribute("href")
        if specProductColor == specProduct:
            print(specProduct)
            wantedProduct = specProduct

driver.get(wantedProduct)

Collectives™ on Stack Overflow

Finding an href link using Python, Selenium, and XPath

HTML source

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

HTML source

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related