Python: How to extract xml embedded in a html file?

Question

I have a html file with xml snipped embedded, the source code is pasted in the pastbin:

<html>
  <head>
    <title> test֤</title>
  </head>
  <body>
    <form name="acsForm" action="" method="post" >
      <textarea rows=10 cols=80 name="xmlText"><?xml version="1.0" encoding="UTF-8"?>
        <samlp:Response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol">
        </samlp:Response> 
      </textarea>
      <textarea name="2nd"> text2....</textarea>             
    </form>
  </body>
</html>

My task is to extract the text enclosed in the first textarea, which is a XML snippet, from the HTML. Without any change to the original snippet. I'm able to get it by using the BeautifulSoup, but it changes all the tag names into lower case.

Amber · Accepted Answer · 2010-04-26 22:35:23Z

1

Try using the BeautifulStoneSoup part of the BeautifulSoup library, which is designed for XML.

answered Apr 26, 2010 at 22:35

Amber

531k89 gold badges643 silver badges558 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

David Z · Accepted Answer · 2010-04-26 22:36:29Z

0

Perhaps lxml would work, although I've never used it myself so I don't know how easy/complicated it would be to do what you want.

answered Apr 26, 2010 at 22:36

David Z

133k29 gold badges264 silver badges284 bronze badges

Comments

bobince · Accepted Answer · 2010-04-26 22:42:08Z

0

(Ugh! Why do so many authors seem to think <textarea> content doesn't need HTML-escaping? Fools!)

Unfortunately BeautifulSoup 3.1 is not applying the (incorrect but common) browser-fixup of treating < and & characters inside <textarea> as text, and is instead creating real XML elements.

BeautifulSoup 3.0 copes with it OK though. Why there's a difference.

answered Apr 26, 2010 at 22:42

bobince

538k111 gold badges675 silver badges846 bronze badges

Comments

georgehu · Accepted Answer · 2010-04-26 22:59:39Z

0

Well I just tried beautifulSoup 3.0, and it doesn't work for me:

xml ='<samlp:Response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"></samlp:Resonse>'
print BeautifulSoup.BeautifulStoneSoup(xml)
<samlp:response xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"

You will notice that the soup has changed Response to response

answered Apr 26, 2010 at 22:59

georgehu

1

Comments

georgehu · Accepted Answer · 2010-04-27 21:53:22Z

0

Finally I found the pyparsing is the best weapon to accomplish the task:

aStart,aEnd = makeHTMLTags("textarea")

search = aStart + SkipTo(aEnd)("body")+ aEnd

saml_resp_str = search.searchString(doc)[0].body relay_state_str = search.searchString(doc)[1].body

answered Apr 27, 2010 at 21:53

georgehu

3004 silver badges12 bronze badges

Collectives™ on Stack Overflow

Python: How to extract xml embedded in a html file?

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related