How to read Values from a web page using python?

Question

I am very very new to python but I have started an internship that requires me to do some python work, I am asked to create an application that can retrieve data from a webpage(IP address) and then compare those values to the correct values and then print out if it has passed or not. Please check this diagram I have made so that you guys can understand it better. Please take a look. Steps needed to execute this application

So far I have only written some python code to check if the website/ip address is up or not but I have no idea how to go further. could you guys please help me to execute the further steps with some examples maybe?

Here is a picture of the website. the values circled in red color need to be compared with the Auxiliary Values I hope this picture helps.

However, I could use http://192.168.100.2/globals.xml on this page to compare the values. Any help is much appreciated.

import requests
import urllib.request
import eventlet
from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd

eventlet.monkey_patch()
with eventlet.Timeout(10):
    print(urllib.request.urlopen("http://192.168.100.5").getcode())
    print("Website is UP") 
    
eventlet.monkey_patch()
with eventlet.Timeout(10):
    print(urllib.request.urlopen("http://10.10.10.2").getcode())
    print("Website is UP")

Akilan Manivannan · Accepted Answer · 2021-03-25 17:46:29Z

2

You are off to a great start! Your next steps should be identifying unique traits about the elements that you want to scrape. Specifically, look for things like class or id names that are unique to only the data that you want to scrape.

You can also use tools like Selector Gadget (https://selectorgadget.com/) that can help automate the process. Unfortunately, since you are accessing local IP addresses, nobody here will be able to help you find these.

After you find the proper selectors, you can use BeautifulSoup to view the data. I'd recommend looking at the find and findall commands that BeautifulSoup has.

answered Mar 25, 2021 at 17:46

Akilan Manivannan

9565 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to read Values from a web page using python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related