Error when getting website table data using python selenium - Multiple tables and Unable to locate element

Question

I am trying to get info from brazilian stock market (BMF BOVESPA). The website has several tables, but my code is not being able to get them.

The code below aims to get all data from table "Ações em Circulação no Mercado" -> one of the last tables from webpage.

I have tried the ones below, but none worked for me:

content = browser.find_element_by_css_selector('//div[@id="div1"]')

and

table = browser.find_element_by_xpath(('//*[@id="div1"]/div/div/div1/table/tbody'))

Thanks in advance for taking my question.

from selenium import webdriver
from time import sleep

url = "http://bvmf.bmfbovespa.com.br/cias-Listadas/Empresas-
Listadas/ResumoEmpresaPrincipal.aspx?codigoCvm=19348&idioma=pt-br"
browser = webdriver.Chrome()
browser.get(url)
sleep(5) #wait website to reload
content = browser.find_element_by_css_selector('//div[@id="div1"]')

HTML can be found at attached picture

As alternative, the code below reaches the same website

url = "http://bvmf.bmfbovespa.com.br/cias-Listadas/Empresas-Listadas/BuscaEmpresaListada.aspx?idioma=pt-br"
Ticker='ITUB4'
browser = webdriver.Chrome()
browser.get(url)
sleep(2)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_txtNomeEmpresa_txtNomeEmpresa_text"]')).send_keys(Ticker)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_btnBuscar"]')).click();
content = browser.find_element_by_id('div1')

Hello All, In the end, the webpage info is shown as iFrame and needs special tools. It is possible to find more information @ elementalselenium.com/tips/3-work-with-frames The solution was solved by @elrich bachman. Tks! — Ricardo
– Ricardo, Commented Mar 14, 2018 at 11:26

elrich bachman · Accepted Answer · 2018-03-14 17:57:11Z

1

Selenium with Python documentation UnOfficial

Hii there

Selenium provides the following methods to locate elements in a page:

find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector

Why your code doesnt work ? because you're not using correct correct code to locate element

you're using xpath inside css selector

content = browser.find_element_by_css_selector('//div[@id="div1"]') #this part is wrong

instead you can do this if you want to select div1

content = browser.find_element_by_id('div1')

here's the correct code

url = "http://bvmf.bmfbovespa.com.br/cias-Listadas/Empresas-

Listadas/BuscaEmpresaListada.aspx?idioma=pt-br"
Ticker='ITUB4'
browser = webdriver.Chrome()
browser.get(url)
sleep(2)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_txtNomeEmpresa_txtNomeEmpresa_text"]')).send_keys(Ticker)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_btnBuscar"]')).click()

I tested it and it worked :)

Mark it as best answer if i helped you :)

edited Mar 14, 2018 at 17:57

answered Mar 5, 2018 at 5:26

elrich bachman

17613 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Ricardo Over a year ago

Elrich, thanks for helping on this. I receive the error: "NoSuchElementException: no such element: Unable to locate element" For me its weird because the element is there. Don't know why selenium does not find the table data. Have it worked for you? Tks again.

elrich bachman Over a year ago

the link you've provided shows this "Essa página não existe ou não pode ser encontrada." so i am not able to figure out whats the problem if you can provide the link i can retry and help you with it .

Ricardo Over a year ago

Thx Elrich, I wiill update code into question. Please see last lines of question (below picture) updated. Thx again

Ricardo Over a year ago

Thx for nth time for your attention, but just to confirm. Where you able to run --- content = browser.find_element_by_id('div1') --- in my new code? I didn't get the table content and received error message

Ricardo Over a year ago

Hey Elrich. Really sorry if I made myself confusing with the question. I did another one, trying to be more precise. You can find it here: stackoverflow.com/questions/49193756/… Tks again

|

Collectives™ on Stack Overflow

Error when getting website table data using python selenium - Multiple tables and Unable to locate element

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related