How to access HTML elements in a nested fashion using Selenium and Python

Question

I'm trying to figure out how to access HTML elements in a nested fashion through Selenium and Python.

For example I have:

box = driver.find_element_by_tag_name('tbody')

which represents the body of the data I'd like to mine. I'd like to iterate through each row in this body (each row characterized by a <tr> tag) using something like:

for driver.find_element_by_tag_name('tr') in box:

But obviously that's not possible because box is a Selenium object and is non-iterable.

What's the best way to do something like this?

undetected Selenium · Accepted Answer · 2022-04-27 14:27:14Z

1

An optimum approach would be to construct locator strategies which would traverse from the parent till the descendants as follows:

Using CSS_SELECTOR:

for element in driver.find_elements(By.CSS_SELECTOR, "tbody tr"):
    print(element.text)

Using XPATH:

for element in driver.find_elements(By.XPATH, "//tbody//tr"):
    print(element.text)

answered Apr 27, 2022 at 14:27

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Matthew Kaplan Over a year ago

Awesome, appreciate the advice!

Matthew Kaplan Over a year ago

Just one more quick question, and I'll be a little more specific this time: I need to access elements td in a row tr in a body tbody. Your method iterates through each element but has no way of distinguishing which row it came from. I need to do something like: for each row in body, create new array, for each element in row, add element to array. And unless I'm not properly understanding your solution, I don't think that's possible with the code you provided. How would I distinguish which elements are part of which row, so I could add them to the appropriate array.

undetected Selenium Over a year ago

@MatthewKaplan I'm afraid. Sounds like a different question all together. Feel free to raise a new question with your new requirement.

Collectives™ on Stack Overflow

How to access HTML elements in a nested fashion using Selenium and Python

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related