Python Selenium. Scraping web page

Question

I want to get the data from the box inside 'Stock Style - Weight' from the url 'https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000NF9P&tab=3' using Selenium

This data is in an iframe. I´m able to switch to the iframe and click the button = 'Weight' but i can´t get the nine figures

Below is my code

driver = webdriver.Chrome(chromedriver)
driver.get("https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000NF9P&tab=3")

iframe = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.XPATH, "//iframe[@id='portfolio']")))
driver.switch_to.frame(iframe)

element1=driver.find_element_by_xpath('/html/body/div/sal-components-pillar-cards-process/div/div[2]/div/div[2]/div[2]')
element2=element1.find_element_by_css_selector("input[type='radio'][value='Weight']").click()

I´ve tried several options

driver.find_element_by_xpath('*//div/div[2]/div/div[2]/div/svg/g/g[3]/g[2]/g[1]/text')

driver.find_element_by_css_selector("mbc-chart-group> g.style-box-text-layer > g:nth-child(1)")

but i get the same error

NoSuchElementException: no such element: Unable to locate element

print out the page source of the webpage using selenium and check whether your xpath/class name matches to yours or not — Ghost Ops
– Ghost Ops, Commented Oct 24, 2021 at 7:50

pmadhu · Accepted Answer · 2021-10-24 11:16:38Z

1

The Elements are in svg and text tags. To access the same you need to use:

//*[local-name()='svg'] or //*[name()='svg']

Link to refer

Xpath for those number would be:

//div[@class='sal-stock-style__weight']//*[name()='svg' and @role='chart']//*[name()='g' and @class='style-box-text-layer']//*[name()='text']

Try like below and confirm:

numbers = driver.find_elements_by_xpath("//div[@class='sal-stock-style__weight']//*[name()='svg' and @role='chart']//*[name()='g' and @class='style-box-text-layer']//*[name()='text']")
for num in numbers:
    print(num.text)

answered Oct 24, 2021 at 11:16

pmadhu

3,4332 gold badges14 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

HamzaFarooq · Accepted Answer · 2021-10-24 09:47:35Z

1

You need to add these two lines to click accept cookies button and investor type button

WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='onetrust-accept-btn-handler']"))).click()
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='btn_professional']"))).click()

Full Code

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000NF9P&tab=3")

WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='onetrust-accept-btn-handler']"))).click()
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='btn_professional']"))).click()

iframe = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.XPATH, "//iframe[@id='portfolio']")))
driver.switch_to.frame(iframe)

element1=driver.find_element_by_xpath('/html/body/div/sal-components-pillar-cards-process/div/div[2]/div/div[2]/div[2]')
element2=element1.find_element_by_css_selector("input[type='radio'][value='Weight']").click()

answered Oct 24, 2021 at 9:47

HamzaFarooq

4391 gold badge4 silver badges18 bronze badges

1 Comment

jccigh Over a year ago

This solves the way i access to the web. I was doing it manually before. But my main problem is how to get the data once I enter to the iframe and the Weight option

Collectives™ on Stack Overflow

Python Selenium. Scraping web page

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related