Parsing href using python

Question

I have web scraping using python that gets this code from the site:

<a href="javascript:document.frmMain.action.value='display_physician_info';document.frmMain.PhysicianID.value=1234567;document.frmMain.submit();" title="For more information, click here.">JOHN, DOE</a>

I want to parse the specific value of href like the value of PhysicianID which is 1234567 inside "document.frmMain.PhysicianID.value"

Currently I'm getting the whole href text something like this:

for i in soup.select('.data'):
     name = i.find('a', attrs = {'title': 'For more information, click here.'})

Any idea? Thanks in advance.

Ofer Sadan · Accepted Answer · 2018-06-04 09:09:03Z

1

Getting in href itself is easy with BeautifulSoup once you've got the link itself:

href = name['href']

Then you can use regex with the re module:

import re
match = re.search(r'document.frmMain.PhysicianID.value=\d*;', href).group()
value = re.search(r'\d+', match).group()
print(value) #prints 1234567

Putting it all together with your code:

import re
for i in soup.select('.data'):
    name = i.find('a', attrs = {'title': 'For more information, click here.'})
    match = re.search(r'document.frmMain.PhysicianID.value=\d*;', href).group()
    value = re.search(r'\d+', match).group()
    print(value) #prints 1234567

answered Jun 4, 2018 at 9:09

Ofer Sadan

12k6 gold badges42 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

SIM · Accepted Answer · 2018-06-04 09:16:47Z

1

Or without regex:

from bs4 import BeautifulSoup

content = """
<a href="javascript:document.frmMain.action.value='display_physician_info';document.frmMain.PhysicianID.value=1234567;document.frmMain.submit();" title="For more information, click here.">JOHN, DOE</a>
"""
soup = BeautifulSoup(content,"lxml")
item = soup.select_one("a")['href'].split("PhysicianID.value=")[1].split(";")[0]
print(item)

Output:

answered Jun 4, 2018 at 9:16

SIM

22.5k6 gold badges45 silver badges116 bronze badges

Collectives™ on Stack Overflow

Parsing href using python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related