Parsing nested elements using selenium not working - python

Question

The picture attached show the structure of the HTML page that I am trying to scrape:

First I retrieve the element league-item and then I am looking for the i item with class name : 'ds-icon-material league-toggle-icon'

Selenium is telling me that it cannot find any item with such name. Here is my code:

path = r"""chromedriver.exe"""
driver = webdriver.Chrome(executable_path=path)
driver.get(_1bet)
time.sleep(5)
#a = driver.find_element_by_class_name('box-content.box-bordered.box-stick.box-bordered-last')

league1 = driver.find_elements_by_class_name('league-list')[0]
league1.find_element_by_class_name("ds-icon-material league-toggle-icon")

Can you please help me? I dont understand why it isn't working.

Thanks

NB: The website I'm scraping is: https://1bet.com/ca/sports/tennis?time_range=all

Prophet · Accepted Answer · 2021-07-24 19:05:43Z

4

I can't access that web page so I can only guess what is going there.
I can figure 2 problems here:

To select element inside element it's better to use XPath starting with a dot .
The element you trying to access having 2 class names. You should use css selector or XPath to locate element according to multiple class names.
So I suggest you trying this:

league1 = driver.find_elements_by_class_name('league-list')[0]
league1.find_element_by_xpath(".//i[@class='ds-icon-material league-toggle-icon']")

edited Jul 24, 2021 at 19:05

answered Jul 24, 2021 at 18:52

Prophet

33.5k28 gold badges58 silver badges90 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

lalaland Over a year ago

btw van you please explain briefly what is the .//. I am trying to fetch another thing based on the same syntax and it is not working

YaDav MaNish Over a year ago

@Prophet can you pls let me now can we use the * instead of . there? Is it have any difference

Prophet Over a year ago

dot on the beginning of XPath means "from here". So if you are searching according to locator //i[@class='ds-icon-material league-toggle-icon'] selenium will search elements matching this locator from the top of the page so if there are several elements matching this locator you will get the first element matching this locator. But if you are trying to locate element inside other element you can use .//i[@class='ds-icon-material league-toggle-icon'] to search inside the HTML starting from league1 web element

Prophet Over a year ago

@YaDavMaNish no, these are different things. * means any. So if we use //*[@class='my_class'] this will match all the elements having class attribute equals my_class value, no matter what tag name is it there div, i, span etc. while dot . at the beginning, before the // or / means from here. Dot can also be in other places, with other meanings ...

Prophet Over a year ago

@lalaland You are welcome. If there are any more questions - you are welcome to ask. Preferably not as a comment rather as a new questions since it can be useful for many other people as well.

|

furas · Accepted Answer · 2021-07-24 18:17:35Z

1

Selenium expects single class name - and it adds dot at the beginning to create CSS selector.
But "ds-icon-material league-toggle-icon" is two classes and it will add dot befor first class but not before second class and this makes proble.

You may use directly css selector with all dots

 .find_element_by_css_selctor(".ds-icon-material.league-toggle-icon")

or you have to trick Selenium and add missing dots between classes

 .find_element_by_class_name("ds-icon-material.league-toggle-icon")

I can't connect with this page to confirm that this is all.

edited Jul 24, 2021 at 18:17

answered Jul 24, 2021 at 17:51

furas

149k12 gold badges121 silver badges171 bronze badges

2 Comments

lalaland Over a year ago

Hello @furas thanks for your answer. I am not sure to understand exactly what you mean. I tried: league1.find_element_by_class_name(".ds-icon-material.league-toggle-icon") and league1.find_element_by_class_name("ds-icon-material.league-toggle-icon") but it doesn't work

furas Over a year ago

As I said I cant acces page and check it - maybe it needs something more. And version with dot at the beginning needs css_selector instead of class_name

Collectives™ on Stack Overflow

Parsing nested elements using selenium not working - python

2 Answers 2

6 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related