Selenium + Python: Print the text attribute of an element

Question

I would like to navigate through a website, find an element and print it.

Python version: 3.10; Selenium Webdriver: Firefox; IDE: PyCharm 2021.3.2 (CE); OS: Fedora 35 VM

I am able to navigate to the appropriate page where the text is generated in a drop down menu. When I locate the element by CSS Selector and attempt to print it, the output does print the text "None". I would like it to print the Plan Name which in this case is "Dual Complete Plan 1". The element is not always present so I also need to catch any exceptions.

The relevant HTML code of the element I am trying to print:

<span class="OSFillParent" data-expression="" style="font-size: 12px; margin-top: 5px;">Dual Complete Plan 1</span>

More of the HTML code of the element I am trying to print (element I am trying to capture is below the fourth div):

<td data-header="Plan Name">
    <div id="b8-b40-l1_0-132_0-$b2" class="OSBlockWidget" data-block="Content.AccordionItem">
    <div id="b8-b40-l1_0-132_0-b2-SectionItem" class="section-expandable  open is--open small-accordion" data-container="" data-expanded="true" aria-expanded="true" aria-disabled="false" role="tab">
        <div id="b8-b40-l1_0-132_0-b2-TitleWrapper" class="section-expandable-title" data-container="" style="cursor: pointer;" role+"button" aria-hidden="false" aria-expanmded="true" tabindex="0" aria-controls="b8-b40-l1_0-132_0-b2-Content" EVENT FLEX
            <div id="b8-b40-l1_0-132_0-b2-Title" class="dividers full-width">
                    <span class="OSFillParent" data-expression="" style="font-size: 12px; margin-top: 5px;">Dual Complete Plan 1</span>
            </div>
        <div class="section-expandable-icon" data-container="" aria-hidden="true"
            ::after
        </div>
    </div>
    <div id="b8-b40-l1_0-132_0-b2-ContentWrapper" class="section-expandable-content no-padding is--expanded" data-container="" tabindex="0" aria-hidden="false" aria-labelledby="b8-b40-l1_0-132_0-b2-TitleWrapper"> 
        <div id="b8-b40-l1_0-132_0-b2-Content" role="tabpanel">
            <a data-link="" href="https://www.communityplan.com" target="_blank" title="Click for more information"> EVENT
                <span class="OSFillParent" data-expression="" style="font-size: 12px;">www.CommunityPlan.com</span>
            </a>
            <span class="OSFillParent" data-expression="" style="font-size: 12px:">Phone Number: 8005224700</span>
        </div>
    </div>
</div>
</div>
</td>

My relevant Selenium code:

# Find the Plan Name & if present set it to the variable "Advantage"
        try:
            Advantage = (WebDriverWait(driver, 5).until(
            EC.presence_of_element_located((By.CSS_SELECTOR, "#b8-b40-l1_0-132_0-b2-Title > span:nth-child(1)"))).get_attribute("value"))
        except:
            pass

print('\033[91;46m', Advantage, '\033[0m')

I expect the output to be "Dual Complete Plan 1", which is what I see on the screen and in the HTML. Instead I get the following:

None

Apparently the "Advantage" variable is being set to "None". Why? I can see the text "Dual Complete Plan 1" that I want to print in the HTML code above. What am I doing wrong? I feel like I need a primer on "get attribute"?

try adding .text() at the end:EC.presence_of_element_located((By.CSS_SELECTOR, "#b8-b40-l1_0-132_0-b2-Title > span:nth-child(1)"))).text() — xtekky
– xtekky, Commented Mar 23, 2022 at 23:11
@xtekky When I add .text() to the end of my code, Advantage = (WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#b8-b40-l1_0-132_0-b2-Title > span:nth-child(1)"))).text()) I get an error code saying NameError: name 'Advantage' is not defined — Auditor
– Auditor, Commented Mar 24, 2022 at 14:18
Maybe try to find the element with (By.CLASS_NAME, 'OSFillParent') — xtekky
– xtekky, Commented Mar 24, 2022 at 17:46

KunduK · Accepted Answer · 2022-03-23 23:16:16Z

2

To get the text Dual Complete Plan 1 you need to use

element.text

or

element.get_attribute("innerHTML")

or

element.get_attribute("textContent")

Instead of presence_of_element_located() use visibility_of_element_located()

and following css selector to identify

div[id*='Title'] > span.OSFillParent

Or

div.dividers.full-width > span.OSFillParent

Code:

        try:
            Advantage = WebDriverWait(driver, 5).until(
            EC.visibility_of_element_located((By.CSS_SELECTOR, "div[id*='Title'] > span.OSFillParent"))).text
        except:
            pass
        print(Advantage )

answered Mar 23, 2022 at 23:16

KunduK

33.4k5 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Auditor Over a year ago

Thank you for the canonical answer. If I right click on “Dual Complete Plan 1” on the webpage in Firefox and right click “inspect”, I see the <span class="OSFillParent" data-expression="" style="font-size: 12px; margin-top: 5px;">Dual Complete Plan 1</span> and when I right click on that and click Copy -> CSS Selector, I get #b8-b40-l1_0-134_0-b2-Title > span:nth-child(1). If I use that CSS Selector in your code, it produces the following error: NameError: name 'Advantage' is not defined..

Auditor Over a year ago

I have a few questions: 1.Why does the CSS Selector copied when I Inspect the webpage not work? 2. How did you determine the CSS Selector was "div[id*='Title'] > span.OSFillParent"? 3.Can you recommend a resource that explains how to determine the proper CSS Selector? 4. Can you recommend a resource that explains element.text and element.get_attribute("textContent") and element.get_attribute("innerHTML")?

KunduK Over a year ago

You can learn css selector from here toolsqa.com/selenium-webdriver/css-selectors-in-selenium

KunduK Over a year ago

It seems copied css selector is dynamic that’s why it is failing. You need to identify element uniquely identified on the page and should not be dynamic.

Auditor Over a year ago

I understand now. The CSS Locator I get by inspecting a webpage may not work if the element is dynamic. And you created the CSS Selector div[id*='Title'] > span.OSFillParent by using the parent of the element that needed to be located: Parent_locator > child_locator. You created the Parent_locator portion by using tag name of the HTML element[attribute_name = ‘attribute_value’] and you created the child_locator by using the class attribute as HTML tag.form-control where the HTML tag is followed by a dot (.) and then the class value. Brilliant.

Collectives™ on Stack Overflow

Selenium + Python: Print the text attribute of an element

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related