Need to extract all links from script tag HTML Python

Question

Basically i need to parse all src="" links from all <script> tags in HTML.

<script src="path/to/example.js" type="text/javascript"></script>

Unfortunately, bs4 cannot do that. Any ideas how can i achieve this?

Of course bs4 can do it, which part of it are you lost on? Finding all script tags or extracting the src attribute? — Alex Hall
– Alex Hall, Commented May 28, 2019 at 12:45
On extracting the src. I've used next code python test = soup.find_all('script') links = [link['src'] for link in test] and recieved an error python KeyError: 'src' This example worked fine with other tags — Razeel
– Razeel, Commented May 28, 2019 at 12:58
Meaning there is at least one script tag without a src attribute, probably has some inline javascript. Try [link['src'] for link in test if 'src' in link] — Alex Hall
– Alex Hall, Commented May 28, 2019 at 13:05

Razeel · Accepted Answer · 2019-05-28 16:11:30Z

4

import requests
import bs4
text = requests.get('http://example.com').text
soup = bs4.BeautifulSoup(text, features='html.parser')
scripts = soup.find_all('script')
srcs = [link['src'] for link in scripts if 'src' in link.attrs]
print(srcs)

edited May 28, 2019 at 16:11

Razeel

471 silver badge6 bronze badges

answered May 28, 2019 at 13:17

Alex Hall

36.2k5 gold badges63 silver badges98 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

QHarr · Accepted Answer · 2019-05-28 17:07:05Z

1

I would condense and use script[src] to ensure script has src attribute

import requests
from bs4 import BeautifulSoup as bs
r = requests.get('http://example.com').content
soup = bs(r, 'lxml') # 'html.parser' if lxml not installed
srcs = [item['src'] for item in soup.select('script[src]')]

answered May 28, 2019 at 17:07

QHarr

84.5k14 gold badges58 silver badges105 bronze badges

Collectives™ on Stack Overflow

Need to extract all links from script tag HTML Python

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related