How do I find HTML Elements not in the page source using Selenium?

Question

So I'm trying to find this <ul> tag I found using inspect element on chrome:

<ul class = "jobs-search-results__list artdeco-list" itemtype="http://schema.org/ItemList"></ul>

This is what I tried in Python:

ul = driver.find_element_by_class_name("jobs-search-results__list artdeco-list")

Which should return the <ul> tag. Instead I get this error:

selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element:{"method":"class","selector":"jobs-search-results__list artdeco-list"}

I get the same error whether I use a tag/xpath/absolutepath selector.

Then I find out this element is not on the HTML page source, and so selenium can't find it. HTML Source (pastebin)

How do I go about finding this element if its not on the page source?

I’m not sure what the issue is here, you simply cannot parse something that doesn’t exist, right? — AMC
– AMC, Commented Dec 12, 2019 at 23:07
So the general problem is the element is there when I use inspect element in the browser, but its not there in the HTML source, so I was just wondering if I can still get to it in the webdriver. — Doobyus
– Doobyus, Commented Dec 12, 2019 at 23:41
It's a bug in find_element_by_class_name. It's a known bug. Just switch to css. — pguardiario
– pguardiario, Commented Dec 13, 2019 at 1:35
Please do not post code as picture. Either you copy paste the code or include link to your test site. This helps people answering the question to reproduce the error easily. — Tek Nath
– Tek Nath, Commented Dec 13, 2019 at 5:25

Tek Nath · Accepted Answer · 2019-12-15 06:04:23Z

1

The class of ul element that you are trying to get is changing while accessing site using Selenium. For this use the xpath as

//ul[contains(@class,'jobs-search__results')]

Now you can find ul element as

ul = driver.find_element_by_xpath("//ul[contains(@class,'jobs-search__results')]")

answered Dec 15, 2019 at 6:04

Tek Nath

1,8213 gold badges23 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do I find HTML Elements not in the page source using Selenium?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related