Python Selenium access inner elements

Question

Just for fun, I try to extract the scheduled games for day 1 of this NFL season. As of now, I have this code in Python

from selenium import webdriver

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

from webdriver_manager.chrome import ChromeDriverManager

browser = webdriver.Chrome(ChromeDriverManager().install())

from selenium.webdriver.chrome.options import Options

opts = Options()
opts.headless=True

#browser = driver(options=opts)
print("Start")

website = "https://www.nfl.com/schedules/2020/REG1"
browser.get(website)
gamedays = WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CLASS_NAME, "nfl-o-matchup-group")))
   
print ("There are: ", len(gamedays))

This prints "There are: 3" which is correct because there are three sections having that class. How can I dig deeper into each section? For example, there is an element having the game date and of course, there are elements having the home team and the away team.

I have tried

   for j in range(0,len(gamedays)):
      game_on = gamedays[j].find_elements_by_class_name('d3-o-section-title')
      print(game_on)

which prints

[<selenium.webdriver.remote.webelement.WebElement (session="d8807c1ca013d7a2d58bd7377b42ca1a", element="034de32d-bf64-4544-94aa-d97ed6640367")>]

That is not helpful.

So, how can I get to that information?

Any help is appreciated.

Try print(game_on.text)

Mike67
– Mike67

2020-09-08 01:27:59 +00:00
Commented Sep 8, 2020 at 1:27 — Mike67
– Mike67, Commented Sep 8, 2020 at 1:27

pygeek · Accepted Answer · 2020-09-08 01:39:39Z

1

Solution

Loop through objects returned in game_on and print their innerHtml

print(game_on.get_attribute("innerHTML"))

OR

for element in game_on:
   print(element.get_attribute("outerHTML"))

answered Sep 8, 2020 at 1:39

pygeek

7,4141 gold badge22 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Michael Over a year ago

The for loop did it. Thank you.

undetected Selenium · Accepted Answer · 2020-09-12 20:34:53Z

1

You were almost there. Using the line of code:

print(game_on)

you were printing the element and you need to print the textContent instead.

Solution

To print the textContent e.g. Friday, September 11th, etc you can use either of the following Locator Strategies:

Using xpath and get_attribute():

browser.get(website)
print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CLASS_NAME, "nfl-o-matchup-group")))])

Using xpath and text attribute:

browser.get(website)
print([my_elem.text for my_elem in WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CLASS_NAME, "nfl-o-matchup-group")))])

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

Outro

Link to useful documentation:

get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

edited Sep 12, 2020 at 20:34

answered Sep 8, 2020 at 12:39

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

3 Comments

Michael Over a year ago

Hm, print(game_on.text) shows AttributeError: 'list' object has no attribute 'text' print(game_on.get_attribute("innerHTML")) shows AttributeError: 'list' object has no attribute 'get_attribute' Thanks for the links.

undetected Selenium Over a year ago

@Michael I assumed the for loop was already in your code trials, so I avoided it. Checkout the updated answer and let me know the status.

Michael Over a year ago

Thanks. I went with the other solution in the meantime.

Collectives™ on Stack Overflow

Python Selenium access inner elements

2 Answers 2

Solution

1 Comment

Solution

Outro

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Solution

1 Comment

Solution

Outro

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related