html table to csv scraper

Question

I'm trying to scrape the table in the following website but was not able to do it:

https://www.moneycontrol.com/financials/relianceindustries/ratiosVI/RI?classic=true#RI

import csv

from bs4 import BeautifulSoup

from urllib.request import urlopen

soup = BeautifulSoup(urlopen('https://www.moneycontrol.com/financials/relianceindustries/ratiosVI/RI?classic=true#RI'))

table = soup.find('table', attrs={ "class" : "table-horizontal-line"})

headers = [header.text for header in table.find_all('th')]

rows = []

for row in table.find_all('tr'):
    rows.append([val.text.encode('utf8') for val in row.find_all('td')])

with open('output_file.csv', 'wb') as f:
    writer = csv.writer(f)
    writer.writerow(headers)
    writer.writerows(row for row in rows if row)

Which part of your code fails? Can you access the site? Can you find the table? Can you extract the rows? Can you write the csv? — G. Anderson
– G. Anderson, Commented Jan 2, 2019 at 19:03
Try doing something = table.find_all('tr')' on a separate line, then go into the loop for row in something:` but that's my best guess without more information on whats wrong — SPYBUG96
– SPYBUG96, Commented Jan 2, 2019 at 19:06
I take it you mean "scrape" and "scraper". Anyway, are you sure "table-horizontal-line" is not only in the code, but a class of the table itself and not for example a row or other tag? I don't see it when I look at the page's code. There might be a better way to identify the table. — Bill M.
– Bill M., Commented Jan 2, 2019 at 19:16

QHarr · Accepted Answer · 2019-01-02 20:05:43Z

1

You can use pandas for this. There are a couple of rows at the top you may wish to remove and replace some other NaNs with empty strings as cleansing.

import pandas as pd
result = pd.read_html('https://www.moneycontrol.com/financials/relianceindustries/ratiosVI/RI?classic=true#RI')
df = result[3].dropna(how='all').fillna('')
df.to_csv(r'C:\Users\User\Desktop\Data.csv', sep=',', encoding='utf-8',index = False )

answered Jan 2, 2019 at 20:05

QHarr

84.5k14 gold badges58 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

html table to csv scraper

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related