Scrape table from webpage into a DataFrame using read_html

Question

I'm trying to do some webscraping but it keeps showing no tables found but there is literally a table on the website. Maybe I'm not writing the right code. Someone help. Here is the link to the website. Film-Locations-in-San-Francisco

error image enter image description here

Welcome to SO! What is the code you have tried so far? It would help your question if you researched how to properly format it. Also, there are plenty of posts out there already about scraping HTML tables in Python. — dcsuka
– dcsuka, Commented Aug 21, 2022 at 2:45
Please provide enough code so others can better understand or reproduce the problem. — Community
– Community Bot, Commented Aug 21, 2022 at 2:58

keramat · Accepted Answer · 2022-08-24 06:56:19Z

1

I guess you are using requests. But you should note that the requests does not receive the table because the table is generated after rendering the page and it is not exists in the requests data. A basic way is to use something like Selenium to first load the page then retrieve the information.

Based on your comment use something similar to:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

driver = webdriver.Chrome(executable_path=r"C:/Users/user/Desktop/chromedriver_win32/chromedriver.exe")
driver.get("https://data.sfgov.org/Culture-and-Recreation/Film-Locations-in-San-Francisco/yitu-d5am/data")
elem = driver.find_element_by_xpath('//*[@id="renderTypeContainer"]/div[4]/div[2]/div/div[4]/div[1]/div/table')
elem.text
#driver.close()

edited Aug 24, 2022 at 6:56

answered Aug 21, 2022 at 3:43

keramat

4,6138 gold badges29 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Divine Over a year ago

This is the code I used i.sstatic.net/5MY75.png

keramat Over a year ago

This does not work? I will try to develop a version with selenium.

keramat Over a year ago

Look at the answer again.

Collectives™ on Stack Overflow

Scrape table from webpage into a DataFrame using read_html

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related