Click through pagination with Selenium in Python

Question

I'm working on building an automation framework using Selenium in Python where I am trying to loop through landing pages on HubSpot.com by clicking on the "Next" button.

The Next button is located in the following HTML according to the Chrome Inspector:

<a class="next" data-reactid=".0.1.0.0.2.1.0.0.1.0.$next.0">
    <span data-reactid=".0.1.0.0.2.1.0.0.1.0.$next.0.1">Next</span>
    <span class="right" data-reactid=".0.1.0.0.2.1.0.0.1.0.$next.0.2"></span>
</a>

And I'm clicking the button using the following Python code:

time.sleep(3)
wait(self.driver, """//a[@class="next" and not(@disabled)]""")
            nextButton = self.driver.find_element(By.XPATH, """//a[@class="next" and not(@disabled)]""")
            hov = ActionChains(self.driver).move_to_element(nextButton)
            hov.click().perform()

where my wait() function is defined as:

def wait(dr, x):
    element = WebDriverWait(dr, 5).until(
        EC.presence_of_element_located((By.XPATH, x))
    )
    return element

This works perfectly fine on the first page. But for some reason when I get to page two, it's not able to click the button. No errors, it just starts doing what it should on page three - but on page two. Also, if I stop the script just before it's supposed to click on the Next button on page two, I can see that the button is highlighted in the Chromedriver.

The Next button is located in the following HTML on page 2 according to the Chrome Inspector::

<a class="next" data-reactid=".0.1.0.0.2.1.0.0.1.0.$next.0">
    <span data-reactid=".0.1.0.0.2.1.0.0.1.0.$next.0.1">Next</span>
    <span class="right" data-reactid=".0.1.0.0.2.1.0.0.1.0.$next.0.2"></span>
</a>

Any suggestions? I am out of ideas.

Does it suddenly start working if you add a sleep between the 1st and 2nd click? — Levi Noecker
– Levi Noecker, Commented Oct 7, 2016 at 14:38
I suppose you use this code two times (one time per page), right?:) Why you use ActionChains instead of simple click()? — Andersson
– Andersson, Commented Oct 7, 2016 at 14:39
@LeviNoecker: I already have a sleep between the clicks. Unfortunately this does not change anything. — mfvas
– mfvas, Commented Oct 7, 2016 at 14:42
@Andersson: I've tried using the click-method as well. No change. — mfvas
– mfvas, Commented Oct 7, 2016 at 14:42
Try using element_to_be_clickable instead of presence_of_element_located as element = WebDriverWait(dr, 5).until(EC.element_to_be_clickable((By.XPATH, x))) and let me know. — Saurabh Gaur
– Saurabh Gaur, Commented Oct 7, 2016 at 16:40

mfvas · Accepted Answer · 2016-10-07 17:21:44Z

1

I have managed to find a workaround. Instead of trying to click the next button I click the relevant page-button. I define j=2 to begin with, and then use the following code:

if self.driver.find_elements(By.XPATH, """//a[@class="next" and not(@disabled)]""") != 0:
                xpathstring = """//div[@class="hs-pagination"]/ul/li/a[text()="%s"]""" % (j)
                waitclick(self.driver, xpathstring)
                nextButton = self.driver.find_element(By.XPATH, xpathstring)
                hov = ActionChains(self.driver).move_to_element(nextButton)
                hov.click().perform()
                time.sleep(3)
                j = j + 1

I still do not understand why the first solution does not work, but this is a working solution to clicking through pagination.

answered Oct 7, 2016 at 17:21

mfvas

1151 silver badge12 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Click through pagination with Selenium in Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related