Finding Specific String Inside HTML Document With Python

Question

I'm working on an idea I had to see if I could find a selection of specific tags on a given website and hit this snag really early on. Below I'm trying to find within a given link's HTML document if there's the presence of something that matches, in this case, 'google-analytics'.

By going to the link, right clicking and viewing the source code there very clearly is something within the site's source code that exactly matches 'google-analytics' but the code below is still not returning something to indicate that is in fact true. Even if I add some condition to show false otherwise, it returns false.

I'm sure the fix is right under my nose but I'm really stuck on this right now. Below is the latest version I've been tinkering with. Previously I had tried simply making the x variable the string itself, x = 'google-analytics' and testing with the same for-loop.

import requests
from bs4 import BeautifulSoup
import re
import csv

URL = "https://www.python.org/"
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
x = soup.find("google-analytics")

#print(soup.prettify())
for line in soup:
    print(line)
    if x in soup:
        print('found')

Andrej Kesely · Accepted Answer · 2020-07-04 22:31:01Z

1

The line x = soup.find("google-analytics") will search for tag <google-analytics>, which doesn't exist in your document - and returns None.

Instead, the string google-analytics is inside <script> tag, so you want to search inside each of these tags.

For example:

import requests
from bs4 import BeautifulSoup

URL = "https://www.python.org/"
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")

for s in soup.select('script'):
    if s.string and 'google-analytics' in s.string:
        print('Found!')
        break
else:
    print('Not Found!')

Prints:

Found!

answered Jul 4, 2020 at 22:31

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Finding Specific String Inside HTML Document With Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related