Python: How can I gather and parse csv data from a webpage?

Question

I have a webpage that prints out csv data using a cgi script. I want to parse the data using Python. So far I know that I can use urllib to request the url and obtain the html into one giant byte string. However, it contains much more than the csv data I need, including html tags, newlines, etc... What I would like to do with this data is to be able to filter rows and columns. This data will eventually go into another csv file which I can use as data to display in graphs (highcharts).

How can I parse the html for just the csv? And is there a library that can gather the csv into a dictionaries or even better, a csv file?

Thanks

Thanks for the suggestion. It looks like Scrapy could definitely work. Unfortunately, this will be a lot more work than I imagined to simply filter rows and columns from a webpage :( — imagineerThat
– imagineerThat, Commented Apr 18, 2013 at 22:22

singer · Accepted Answer · 2013-04-18 23:11:40Z

1

Try

1) Use urlib as you metioned

2) Use Beautiful soup for geting a part of document you need

3) Use standard csv parser or pandas to parse data you received at the previous step

edited Apr 18, 2013 at 23:11

answered Apr 18, 2013 at 22:51

singer

2,63625 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python: How can I gather and parse csv data from a webpage?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related