Python and BeautifulSoup find text string in html

Question

Have the following function that checks a certain webpage for a keywoard

def checkString():   
    url_a = 'https://launchstudio.bluetooth.com/ListingDetails/50756'
    r_a = requests.get(url_a)
    soup_a = BeautifulSoup(r_a.text)

    for blem in soup_a(text=re.compile(r'RFCOMM')):
        return True

    return False

Have verified that my soup_a is the same as the view-source of the url, but it seems that my search will only return results contain within the head tags and have a hard time figuring out why. Any suggestions?

Python version 2.7.5

Does the page source show the RFCOMM string as one, not e.g. <b>RF</b>comm or RF<wbr/>COMM? — 9000
– 9000, Commented Dec 28, 2017 at 20:58

Ajax1234 · Accepted Answer · 2017-12-28 21:00:44Z

2

You need to pass lxml to the BeautifulSoup class. Also, return True will break out of the for-loop if a match is found. Thus, if RFCOMM is indeed found in the head tags, the loop will quit and no more matches will be registered. It may be better to use a list comprehension and determine if any matches are found:

from bs4 import BeautifulSoup as soup
import urllib.request as urllib
import re
def checkString():   
   url_a = 'https://launchstudio.bluetooth.com/ListingDetails/50756'
   s = soup(str(urllib.urlopen(url_a).read()), 'lxml')
   return bool([i for i in s(text=re.compile(r'RFCOMM'))])

print(checkString())

Output:

True

answered Dec 28, 2017 at 21:00

Ajax1234

71.7k9 gold badges67 silver badges110 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

lillahimmel Over a year ago

Thanks - do not care if it was found inside the heads tag, but for some reason it would not allow me to find anything outside the head tags. Using "html.parser" as argument for the BeautifulSoup did the trick for me for some reason (guess similar to what lmxl would?)

Collectives™ on Stack Overflow

Python and BeautifulSoup find text string in html

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related