Parse specific value from internet page using python

Question

I am trying to parse a value called secu from pages in the format like: https://www.ariva.de/[WKN-input]/historische_kurse

I can find in looking at the source code of the page:

<input type="hidden" name="secu" value="717816" />

E. g.: https://www.ariva.de/A0D9PT/historische_kurse

In the end I would like to store it to a CSV. This is the code I already have, the parse part is missing:

import pandas as pd


# read WKN names from CSV
wkn_list=pd.read_csv('all_wkn_list.csv')

df_output = pd.DataFrame(columns=['wkn', 'secu'])

# loop through WKNs
i=0
for index, row in wkn_list.iterrows():
    print(row[0])
    url = 'https://www.ariva.de/'+str(row[0])+'/historische_kurse'
    
    # parse secu 
    secu_parsed = "test"
    
    # store wkn and secu
    df_output.loc[i] = [str(row[0])] + [str(secu_parsed)]
    i=i+1

# store WKN + secu to CSV    
df_output.to_csv('output.csv')

Use requests to download the HTML then use beautifulsoup to parse the HTML and extract the value. — user3064538
– user3064538, Commented Jan 19, 2021 at 20:43

Matthias · Accepted Answer · 2021-01-19 20:54:17Z

1

You can use the libraries requests and BeautifulSoup for that. Your code would look like this:

import requests
from bs4 import BeautifulSoup

page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
input_element = soup.find('input')

There are more ways to find elements based on their tag, class, id... To know more about that, take a look at the BeautifulSoup-documentation.

answered Jan 19, 2021 at 20:54

Matthias

413 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

StefanOverFlow Over a year ago

Added 'secu_parsed = soup.find("input", {"name":"secu"})['value']' and it works.

Collectives™ on Stack Overflow

Parse specific value from internet page using python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related