Python Selenium: WebDriverWait, element not found despite loading

Question

I'm trying to load a URL which has dynamic HTML. The selenium webdriver loads the URL fine, but it appears to be searching for elements before the HTML has fully rendered. I tried WebDriverWait, but that does not appear to work.

Interestingly, after selenium fails to find the element, I can manually use the Chrome developer tools to find the element successfully.

Any ideas?

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait

chromepath = "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chromedriver.exe"
url        = "https://www.msci.com/end-of-day-data-search"
delay      = 15 #seconds

br = webdriver.Chrome(chromepath)
br.get(url)

try:
    br.find_element_by_class_name("accept-btn").click()
    print("accepted terms and conditions")
finally:
    try:
        labelxpath   = "//*[@id='form-content']/table[1]/tbody/tr[2]/td/span"
        labelElement = WebDriverWait(br, delay).until(lambda br: br.find_element_by_xpath(labelxpath))
        print(labelElement)
    except:
        print("could not find label")
        br.close()

Script results: "accepted terms and conditions" "could not find label"

alecxe · Accepted Answer · 2015-10-01 04:17:11Z

2

Actually, it's not only about waiting for the page to load.

The element you are looking for is inside an iframe. You need to switch to it's context before searching for the element:

driver.switch_to.frame(0)  # 0 here is an index - using 0 since there is only one iframe

answered Oct 1, 2015 at 4:17

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sayjota Over a year ago

Thank you that fixed everything

Collectives™ on Stack Overflow

Python Selenium: WebDriverWait, element not found despite loading

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related