How to extract a dynamic substring from a list of strings in Python?

Question

I have a list of strings I scraped off the internet and I'm looking to extract their 'href':

<li class="subnav__item"><a class="subnav__link " href="/red-wine">Red Wine</a></li>
<li class="subnav__item"><a class="subnav__link " href="/white-wine">White Wine</a></li>
<li class="subnav__item"><a class="subnav__link " href="/rose-wine">Rosé Wine</a></li>
<li class="subnav__item"><a class="subnav__link " href="/fine-wine">Fine Wine</a></li>

For example, I'm looking to loop through the list and dynamically extract

/red-wine

from

<li class="subnav__item"><a class="subnav__link " href="/red-wine">Red Wine</a></li>

Thanks!

use the reference : stackoverflow.com/questions/4666973/… This might be what you are looking for.. — Samarth
– Samarth, Commented Jan 17, 2018 at 7:38

Adeel Ahmad · Accepted Answer · 2018-01-17 07:39:31Z

1

You can also get the required text using Beautiful Soup:

from bs4 import *
data = '\
<li class="subnav__item"><a class="subnav__link " href="/red-wine">Red Wine</a></li>\
<li class="subnav__item"><a class="subnav__link " href="/white-wine">White Wine</a></li>\
<li class="subnav__item"><a class="subnav__link " href="/rose-wine">Rosé Wine</a></li>\
<li class="subnav__item"><a class="subnav__link " href="/fine-wine">Fine Wine</a></li>'
soup = BeautifulSoup(data, "html.parser")

lis = soup.findAll('a')
for li in lis:
    print(li['href'])

/red-wine
/white-wine
/rose-wine
/fine-wine

answered Jan 17, 2018 at 7:39

Adeel Ahmad

1,0531 gold badge11 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Gozy4 · Accepted Answer · 2018-01-17 07:37:01Z

1

You can use lxml for this. Something like this:

from lxml import html
import request

response = request.get('<your url>')
tree = html.fromstring(response.text)
href = tree.xpath('//a[@class="subnav__item"]/@href')

This should get you all the href in from the class "subnav__item"

answered Jan 17, 2018 at 7:37

Gozy4

5147 silver badges13 bronze badges

Collectives™ on Stack Overflow

How to extract a dynamic substring from a list of strings in Python?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related