selenium.common.exceptions.TimeoutException locating an element using Selenium with Python

Question

I'm trying to crawl a page that loads certain data using js request using selenium so I used "visibility_of_element_located" with a timeout. however, even setting it for 15 seconds doesn't work and I get a timeout error.

url = '''https://ukranews.com/en/news/850009-u-s-believes-russia-s-war-with-ukraine- 
will-last-until-end-of-2022-cnn'''        
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.page_load_strategy = "none"
browser = webdriver.Chrome(options=options,
                           executable_path='C:\\Users\\313-SsS\\Downloads\\Compressed\\chromedriver_win32\\chromedriver.exe',
                           )
browser.get(url)
timeout_in_seconds = 5
WebDriverWait(browser, timeout_in_seconds).until(ec.visibility_of_element_located((By.CLASS_NAME, 'article_content')))
browser.execute_script("window.stop();")
html = browser.page_source

Here is the error I got:

selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
Backtrace:
Ordinal0 [0x008E7413+2389011]

I also use the same code for another website that also loads the content using js and it works just fine.

undetected Selenium · Accepted Answer · 2022-04-16 19:11:47Z

1

There are atleast 8 elements which is identified by the locator strategy you have used:

WebDriverWait(browser, timeout_in_seconds).until(ec.visibility_of_element_located((By.CLASS_NAME, 'article_content')))

8 elements

Solution

To wait for the visibility of the <h1> element with text as U.S. Believes Russia's War With Ukraine Will Last Until End Of 2022 - CNN you can use either of the following locator strategies:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.article_content h1")))

Using XPATH:

WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='article_content']//h1")))

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

answered Apr 16, 2022 at 19:11

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

mohsen amini Over a year ago

still have the same problem. after 20 seconds I got selenium.common.exceptions.TimeoutException error.

undetected Selenium Over a year ago

Remove --headless and retest and let me know what you see.

mohsen amini Over a year ago

it works now. Thanks a lot. but how can I stop it from opening the browser now?

undetected Selenium Over a year ago

@mohsenamini Sounds to be a different question all together. Feel free to raise a new question as per your new requirement.

Collectives™ on Stack Overflow

selenium.common.exceptions.TimeoutException locating an element using Selenium with Python

1 Answer 1

Solution

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Solution

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related