Capture elements with Selenium which are loaded lazy in Python

Question

Trying to click a button with Selenium, but I keep getting an error:

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"AGREE"}

Here's the button I am trying to click.

I assume, the popup is loaded lazy. I found some sources, but could not make it work. Here's my code

import pandas as pd
import bs4
import selenium as sel
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
import time
import os

driver = sel.webdriver.Chrome() #uses Chrome driver in usr/bin/ from https://chromedriver.chromium.org/downloads
url = 'https://fbref.com/en/squads/0cdc4311/Augsburg'
driver.get(url)
time.sleep(5)
html = driver.page_source
soup = bs4.BeautifulSoup(html, 'html.parser')

#click button -> accept cookies
element = driver.find_element(By.LINK_TEXT, "AGREE")
element.click()

>>> NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"AGREE"}

I also tried

[...]
driver = sel.webdriver.Chrome() #uses Chrome driver in usr/bin/ from https://chromedriver.chromium.org/downloads
driver.implicitly_wait(10) # seconds'
[...]

and the popup is definitively there. But still get the same error.

Mhhh, probably some geographical specific problem to reproduce? I added a screenshot and a missing line of code. — AndreasInfo
– AndreasInfo, Commented Dec 16, 2021 at 10:53

HedgeHog · Accepted Answer · 2021-12-16 13:34:18Z

1

What happens?

You try to select a <button> via .LINK_TEXT, "AGREE", what won't work, cause it is a <button> not a link.

How to fix?

Wait for the <button> selected by xpath to be clickable:

#click button -> accept cookies
element = WebDriverWait(driver, 3).until(EC.element_to_be_clickable((By.XPATH, '//button[text()="AGREE"]')))
element.click()

answered Dec 16, 2021 at 13:34

HedgeHog

25.4k5 gold badges18 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

AndreasInfo Over a year ago

Found out a couple of minutes ago. But thanks anyway.

AndreasInfo Over a year ago

Would you help me one more time? If I do element = driver.find_element(By.XPATH, '//button[text()="Get table as CSV (for Excel)"]') and element.click() I get Message: element not interactable

HedgeHog Over a year ago

Of course SO is happy to help you with this, but to keep question / answer clean - This would be predestined for asking a new question Thanks

AndreasInfo Over a year ago

stackoverflow.com/questions/70381289/…

Collectives™ on Stack Overflow

Capture elements with Selenium which are loaded lazy in Python

1 Answer 1

What happens?

How to fix?

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

What happens?

How to fix?

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related