How to click a label element utilizing Python and Selenium

Question

I have been building a webscrapper bot with Python and Selenium, but have run into an issue. The website I am using to scrape information has a fieldset html tag with 4 label tags. I am specifically looking to click one of these label tags, but all of them have the same class name. I was hoping to see if anyone had some insight on how I can work around this issue. I have tried out a couple of things but unfortunately none are working.

This is the HTML code:

<fieldset>
                            <legend class="sr-only">Are you a?</legend>
                            <input type="hidden" name="_token" value="Nw3CU75ZNiiwSvcXofVUm3hOpcFkvzEc8UdjlPcX">
                            <span id="errMsg"></span>
                            <div aria-live="assertive" class="error-message"></div>
                            <label class="custom-control custom-radio" tabindex="0" for="radio1">
                                <input tabindex="-1" id="radio1" name="radioStep1" type="radio" value='in-network'
                                    class="custom-control-input" aria-describedby="in-network-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id='in-network-balloon'>
                                    Staying In-Network <span class='sr-only'>If your provider takes your insurance</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Staying In-Network</h3>
                                        <p>If your provider takes your insurance</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                            <label class="custom-control custom-radio" tabindex="0" for="radio2">
                                <input tabindex="-1" id="radio2" name="radioStep1" type="radio" value='out-network'
                                    class="custom-control-input" aria-describedby="out-network-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id='out-network-balloon'>
                                    Going Out-of-Network <span class='sr-only'>If your provider does not take your insurance.</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Going Out-of-Network</h3>
                                        <p>If your provider does not take your insurance.</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                            <label class="custom-control custom-radio selected" tabindex="0" for="radio3">
                                <input checked tabindex="-1" id="radio3" name="radioStep1" type="radio" value='not-sure'
                                    class="custom-control-input" aria-describedby="not-sure-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id="not-sure-balloon">
                                    Not Sure <span class='sr-only'>If you don't know if your provider takes your insurance</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Not Sure</h3>
                                        <p>If you don't know if your provider takes your insurance</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">
                                        Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                            <label class="custom-control custom-radio" tabindex="0" for="radio4">
                                <input tabindex="-1" id="radio4" name="radioStep1" type="radio" value='uninsured'
                                    class="custom-control-input" aria-describedby="uninsured-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id="uninsured-balloon">
                                    Uninsured <span class="sr-only">If you don't have health insurance or if your insurance pays for providers who aren’t in your plan.</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Uninsured</h3>
                                        <p>If you don't have health insurance or if your insurance pays for providers who aren’t in your plan.</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">
                                        Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                        </fieldset>

Here is an image of what the page looks like. I am specifically looking to press the "Not Sure" section, and then "next"

I have tried this code

    label_elements = driver.find_elements_by_css_selector("custom-control custom-radio")
    label_elements[1].click()

along with this

label_elements = driver.find_elements_by_css_selector("custom-control custom-radio")

for label_element in label_elements:
    label_element.click()

and a couple different variations of the above code, but it still has not been working.

Here is my full code:

import selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
import time
import os
from selenium.webdriver.common.action_chains import ActionChains

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

s=Service('/Users/[name]/Desktop/chromedriver')

chromeOptions = Options()
chromeOptions.headless = False
driver = webdriver.Chrome(service=s, options=chromeOptions)


list_data = []

def initalize_browser():
    driver.get("https://www.fairhealthconsumer.org/")
    driver.maximize_window()
   
    driver.find_element(By.XPATH, '//button[@class="mx-auto green-btn btnHref"]').click()
    
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='radio1' and normalize-space()='Einloggen']"))).click()
    driver.find_element(By.ID, 'not-sure-balloon').click()
    driver.find_elements(By.CLASS_NAME, 'fa-chevron-square-right')[2].click()

    print(driver.page_source)

initalize_browser()


driver.quit

Here is the link: fairhealthconsumer.org/medical

Any guidance is super appreciated, thank you!

Of course, here it is fairhealthconsumer.org/medical

imherebcsomethingisntworking
– imherebcsomethingisntworking

2022-12-18 03:49:29 +00:00
Commented Dec 18, 2022 at 3:49 — imherebcsomethingisntworking
– imherebcsomethingisntworking, Commented Dec 18, 2022 at 3:49

AbiSaran · Accepted Answer · 2022-12-18 04:22:16Z

1

Try this:

list_data = []
    
def initalize_browser():
    driver.get("https://www.fairhealthconsumer.org/")
    driver.maximize_window()

    driver.find_element(By.XPATH, '//button[@class="mx-auto green-btn btnHref"]').click()

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, ".//*[@for='radio3']/span"))).click()
    driver.find_element(By.XPATH, ".//*[@for='radio3']//span[@class='pt-2 border-top border-white']").click()
    print(driver.page_source)

initalize_browser()

edited Dec 18, 2022 at 4:22

answered Dec 18, 2022 at 3:58

AbiSaran

2,6983 gold badges13 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

imherebcsomethingisntworking Over a year ago

For some reason that's not working! I tried some variation of that before as well.

AbiSaran Over a year ago

What the error you've got?

imherebcsomethingisntworking Over a year ago

Traceback (most recent call last): File "/Users/[name]/Desktop/python-scrapper-bot/sample.py", line 50, in <module> initalize_browser() File "/Users/[name]/Desktop/python-scrapper-bot/sample.py", line 33, in initalize_browser WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='radio1' and normalize-space()='Einloggen']"))).click() File "/Users/[name]/Library/Python/3.9/lib/python/site-packages/selenium/webdriver/support/wait.py", line 90, in until raise TimeoutException(message, screen, stacktrace) Line 50 = initalize_browser()

imherebcsomethingisntworking Over a year ago

I have updated my post with my full current code!

AbiSaran Over a year ago

Updated the code, check it, it is working fine.

Andrew Ryan · Accepted Answer · 2022-12-18 04:03:49Z

0

why not use this for the 'not sure' button:

driver.find_element_by_id('not-sure-balloon').click()

and then this for the 'next' button

driver.find_elements_by_class_name('fa-chevron-square-right')[2].click()

Though this is usually the recommended way:

from selenium.webdriver.common.by import By
...
driver.find_element(By.ID, 'not-sure-balloon').click()
driver.find_elements(By.CLASS_NAME, 'fa-chevron-square-right')[2].click()

edited Dec 18, 2022 at 4:03

answered Dec 18, 2022 at 3:56

Andrew Ryan

1,6313 gold badges17 silver badges24 bronze badges

1 Comment

imherebcsomethingisntworking Over a year ago

Hello, thank you so much for the suggestion. For some reason this is not working either. I have added my full code to my post, maybe I formatted something incorrectly? I would assume what you said should work perfectly

Leo · Accepted Answer · 2022-12-18 04:10:15Z

0

The id of the not sure button is: not-sure-balloon

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service


s = Service("D:\chromedriver_win32\chromedriver.exe")
browser = webdriver.Chrome(service=s)
url = "https://fairhealthconsumer.org/medical"

browser.get(url)


browser.find_element(By.ID, 'not-sure-balloon').click()

edited Dec 18, 2022 at 4:10

answered Dec 18, 2022 at 4:04

Leo

4042 silver badges6 bronze badges

1 Comment

Andrew Ryan Over a year ago

they are looking to press the "Not Sure" section, and then "next"

Collectives™ on Stack Overflow

How to click a label element utilizing Python and Selenium

3 Answers 3

5 Comments

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related