Parsing Python Response using httplib

Question

After connecting to a socket and capturing the response using .read() how do I parse the input stream and read lines?

I see the data is returned without any CRLF

<html><head><title>Apache Tomcat/6.0.16 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 404 - /index.html</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>/index.html</u></p><p><b>description</b> <u>The requested resource (/index.html) is not available.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.22</h3></body></html>

Maybe the read() is not working correctly yet as the output seems to be an error message — John La Rooy
– John La Rooy, Commented Feb 2, 2010 at 4:21
Ok, so it seems like httplib and urllib return different data. urllib worked as expected so thanks for the tip. Whoever mentioned that, urllib.read() returns the data with CRLF where as httplib.read() seems to return the data deleting the whitespace. — Dave
– Dave, Commented Feb 3, 2010 at 18:59

Bill the Lizard · Accepted Answer · 2012-08-03 14:17:53Z

3

You have to parse the HTML. Python has several ways of parsing HTML - one of them the built-in HTMLParser module. Another, and probably better way, is the 3rd party BeautifulSoup module.

Many other issues dealing with HTML processing are explained in this nice article. You can also read the relevant chapter of the (free online) Dive into Python book.

edited Aug 3, 2012 at 14:17

Bill the Lizard

407k213 gold badges579 silver badges892 bronze badges

answered Feb 2, 2010 at 4:58

Eli Bendersky

276k92 gold badges371 silver badges427 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

danben · Accepted Answer · 2010-02-02 04:16:26Z

0

Use an HTML parser. Beautiful Soup seems to be a popular one.

answered Feb 2, 2010 at 4:16

danben

83.8k18 gold badges126 silver badges149 bronze badges

Collectives™ on Stack Overflow

Parsing Python Response using httplib

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related