clicking button with python/selenium/phantomjs

Question

I'm trying to scrape this with a combination of python 3.5/selenium/phantomjs.

There is a button which loads more offers

<button data-behavior="result-paging" class="button button-text--centered">
    Mehr laden
</button>

This exact button is twice inside the HTML-Code and if all offers are listed the first button looks as follows, while the second button stays the same:

<button data-behavior="result-paging" class="button button-text--centered is-hidden">
   Mehr laden
</button>

To click the first button I tried:

    while True:
        try:
            time.sleep(4)
            btnElements = driver.find_elements_by_xpath("//button[@data-behavior='result-paging']")
            for btnElement in btnElements:
                btnElement.click()
                # btnElement.send_keys("\n")
                if i==11:
                    break
                else:
                    i=i+1
        except:
            break

and

    while True:
        try:
            time.sleep(4)
            elements= driver.find_elements_by_xpath("//button[@data-behavior='result-paging']")
            driver.execute_script("arguments[0].click();", elements[0])
            if i==11:
                break
            else:
                i=i+1
        except:
            break

The loop was just there for quick testing purposes since the script would run eternally otherwise. With booth approaches no new content was loaded, so clicks werent working.

Does anybody have an idea how to solve this issus (perform clicks and load other offers)?

Edit:

Apparently this problem is caused by phantomjs, since I could run the same script with chromedriver. Unfortunately I need to run phantomjs. Did anybody experienced this behaviour before?

You can try this xpath to click first button :- (//button[@data-behavior='result-paging'])[1] — Ankur Singh
– Ankur Singh, Commented Apr 23, 2018 at 13:59
Try to wait before click event.. What error you are getting ? — Ankur Singh
– Ankur Singh, Commented Apr 23, 2018 at 14:15

undetected Selenium · Accepted Answer · 2018-04-23 19:00:52Z

As per your code block the Locator Strategy which you have used as ...

find_elements_by_xpath("//button[@data-behavior='result-paging']")

... doesn't identifies the visible button with text as Mehr laden uniquely.

Solution

To identify the visible button with text as Mehr laden uniquely you can use eith of the following Locator Strategies :

CSS_SELECTOR :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
# lines of code
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[data-psa-scope=tarifflist] button.button.button-text--centered"))).click()

XPATH :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
# lines of code
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@data-psa-scope='tarifflist']//button[@class='button button-text--centered']"))).click()

Collectives™ on Stack Overflow

clicking button with python/selenium/phantomjs

1 Answer 1

Solution

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Solution

Comments

Your Answer

Sign up or log in

Post as a guest

Related