Converting html table to a pandas dataframe

Question

I have been trying to import a html table from a website and to convert it into a pandas DataFrame. This is my code:

import pandas as pd
table = pd.read_html("http://www.sharesansar.com/c/today-share-price.html")
dfs = pd.DataFrame(data = table)
print dfs

It just displays this:

0       S.No                                     ...

But if I do;

for df in dfs:
    print df

It outputs the table..

How can I use pd.Dataframe to scrape the table?

Mohammad Yusuf · Accepted Answer · 2017-02-09 05:42:11Z

5

HTML table on the given url is javascript rendered. pd.read_html() doesn't supports javascript rendered pages. You can try with dryscrape like so:

import pandas as pd
import dryscrape

s = dryscrape.Session()
s.visit("http://www.sharesansar.com/c/today-share-price.html")
df = pd.read_html(s.body())[5]
df.head()

Output:

answered Feb 9, 2017 at 5:42

Mohammad Yusuf

17.1k12 gold badges60 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Converting html table to a pandas dataframe

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related