How to convert raw html from the web into parsable xml in Python

Question

I thought BeautifulSoup could do that, but it does not seem to do the trick.

What method have you already used, and is long term reliable ?

Did you try utidylib.berlios.de ? I don't know it, but Tidy is able to convert ugly HTML into sexy XML. Maybe its python wrapper can do it too ? — Scharron
– Scharron, Commented Sep 13, 2010 at 15:10

supakeen · Accepted Answer · 2010-09-13 15:37:51Z

4

You could use the lxml library, specifically lxml.html which gives you an ETree object which you can then serialize as XML with (amongst others) the .tostring() method.

If this fails on your HTML (it is too broken) you can use ElementSoup (an extension on BeautifulSoup) to build a lxml.html tree.

answered Sep 13, 2010 at 15:37

supakeen

2,94419 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Scharron · Accepted Answer · 2010-09-13 16:10:40Z

2

You can try http://utidylib.berlios.de/ , a python wrapper for tidy library. Tidy works well in most cases.

For something more robust (or at least more browser-like), I guess you could try webkit or gecko. I'm not sure the wrappers responsible for cleaning HTML are available, but you can have a look.

answered Sep 13, 2010 at 16:10

Scharron

17.9k6 gold badges46 silver badges63 bronze badges

Collectives™ on Stack Overflow

How to convert raw html from the web into parsable xml in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related