python regular expression find and replace html tag with specific attribute value

Question

Am trying to write a regular expression in python that would find all img tags where src attribute equal to a specific value.. I tried to write the below

   # where thm equal /public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82
   p = re.compile(r'<img.*?%s.*?>' % thm)
   print p.pattern
   print p.sub(linked_image, c)

and below what I got as an output

<img.*?/public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82.*?>

<p><img src="/public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82" alt=""></p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf </p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf 
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf

what about using BeautifulSoup for this job? crummy.com/software/BeautifulSoup — Totem
– Totem, Commented Dec 15, 2013 at 15:12
I could use the beautifulSoup, but am altering content that is unstructured and it should maintain its structure without change. problem with beautifulSoup, I won't be able to output the processed soup to the previous structure as output... that's why i need to change replace string method — Mo J. Mughrabi
– Mo J. Mughrabi, Commented Dec 15, 2013 at 20:33
Please take a look at the new version - I found the error in the raw thm variable and proposed two solutions below. — Jon
– Jon, Commented Dec 15, 2013 at 21:30

Jon · Accepted Answer · 2013-12-17 08:57:56Z

A solution with LXML

To compare a solution with Regular expressions and LXML I created another post:

An easier and more stable solution is to use lxml with etree. In that solution you access certain DOM elements and edit them.

Convert the HTML string and get it through the right xpath e.g. .//img. The xpath returns a list of all found elements where you can get and set the src attribute. The function etree.tostring(tree) returns an edited string:

from lxml import etree
tree = etree.HTML('''<html>
                     <body>
                        <h1>Title</h1>
                        <img src="/media/old/another_logo.png" alt="" />
                        <p>Lorem Ipsum</p>
                        <p><img src="/media/old/logo.png" alt=""/></p>
                     </body>
                  </html>''')

imgs = tree.xpath('.//img')

for img in imgs:
    print 'OLD_SOURCE', img.get('src')
    img.set('src', '/media/new/python.jpg')

print etree.tostring(tree)

Output

OLD_SOURCE /media/old/another_logo.png
OLD_SOURCE /media/old/logo.png

<html>
    <body>
        <h1>Title</h1>
            <img src="/media/new/python.jpg" alt=""/>
            <p>Lorem Ipsum</p>
            <p><img src="/media/new/python.jpg" alt=""/></p>
    </body>
</html>

Community · Accepted Answer · 2017-05-23 12:03:44Z

A solution with Regular Expressions:

I realized that the string which was inserted into thm was not escaped. So before adding it to your regular expression you need to escape all symbols with an additional signification in the Regular Expression language - here, ? and ..

I replaced ? with [?]{1} and . with \.. The resulting regular expressions matches now the test string.

import re
thm = '/public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82'

all_html_code = '''<img.*?/public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82.*?>

<p><img src="/public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82" alt=""></p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf </p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf
</p><p>lksj lksdfj lsdkfj sldkfj sldkfj lskdfj lsjf lksjf lksj flksdjf klsj flk dkj sdlkfj sdlkfj sldkjf sldkfj lsdkjf lskjflsjfsl lksdjf '''

escaped_thm = thm.replace('.', '\.').replace('?','[?]{1}')
p = re.compile(r'<img.*?src="(%s)".*?>' % escaped_thm)

test_img = '''<img src="/public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82" alt="">'''
print p.match(test_img)

new_img_tag = '<img src="/python/logo.jpg" alt=""/>'
print p.sub(new_img_tag, all_html_code)

By the way, why are you looking for <img src=""...>? You could replace the src attribute directly:

escaped_thm = thm.replace('.', '\.').replace('?','[?]{1}')
p = re.compile(r'src="(%s)"' % escaped_thm)

replacement = '''src="/python/logo.jpg"'''
print p.sub(replacement, all_html_code)

Output 1

<_sre.SRE_Match object at 0x... >
<img.*?/public_media/cache/84/b5/84b59e293cbdb7041b68a84977d62cf3.jpg?image_pk=82.*?>

<p><img src="/python/logo.jpg" alt=""/></p><p>lksj lksdfj ... lksdjf

Output 2

<p><img src="/python/logo.jpg" alt=""></p><p>lksj lksdfj ... lksdjf

After asking about a proper way to escape symbols of regular expressions (Regular expression to escape regular expressions) - I can recommend an re.escape instead of the two replace methods.

Using LXML

Do you need to use regular expressions? Regular expressions for HTML can be highly problematic. See more information and wonderful post here(RegEx match open tags except XHTML self-contained tags).

I would rather use XPath like here.

I could use the beautifulSoup, but am altering content that is unstructured and it should maintain its structure without change. problem with beautifulSoup, I won't be able to output the processed soup to the previous structure as output... that's why i need to change replace string method
I added a solution with regex. Please check out the example above.

Collectives™ on Stack Overflow

python regular expression find and replace html tag with specific attribute value

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related