selenium css selector or xpath for complex class doesn't work when run as script

Question

The following code, which extracts elements using css selector, works in the ipython3 terminal, but doesn't find the elements when run as script:

from selenium import webdriver 
driver = webdriver.Chrome()

url = scrape_url + "&keywords=" + keyword
driver.get(url)
driver.find_elements_by_css_selector(".search-result.search-result__occluded-item.ember-view")

The complex class of the element:

"search-result search-result__occluded-item ember-view"

The following xpath worked in the terminal, but not as a script:

driver.find_elements_by_xpath("//li[contains(@class, 'search-result search-result__occluded-item')]")

Andersson · Accepted Answer · 2017-03-02 11:49:34Z

0

This might be a timing issue: required element could be generated dynamically, so you need to wait some time until it appears in DOM:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver 
driver = webdriver.Chrome()

url = scrape_url + "&keywords=" + keyword
driver.get(url)
wait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//li[contains(@class, 'search-result search-result__occluded-item')]")))

Also some class names could be assigned dynamically. That's why using compound name as "search-result search-result__occluded-item ember-view" might not work without ExplicitWait

answered Mar 2, 2017 at 11:49

Andersson

52.8k18 gold badges83 silver badges132 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2017-05-23 10:29:59Z

0

If you can't find any elements with selenium css selector, then can you always try to use xpath instead of the css selector.

More information about that can be found here.

edited May 23, 2017 at 10:29

CommunityBot

11 silver badge

answered Mar 2, 2017 at 11:26

D. J.

7046 silver badges25 bronze badges

1 Comment

mrbot Over a year ago

xpath did not work either. Sorry, I will update the question

Chanda Korat · Accepted Answer · 2017-03-02 11:28:37Z

0

Pass only partial class name like,

driver.find_elements_by_css_selector(".search-result__occluded-item")

answered Mar 2, 2017 at 11:28

Chanda Korat

2,5812 gold badges22 silver badges24 bronze badges

1 Comment

Chanda Korat Over a year ago

Put wait before finding element.

Collectives™ on Stack Overflow

selenium css selector or xpath for complex class doesn't work when run as script

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related