How to find text of a specific cell from a html table (on a url) using python?

Question

from selenium import webdriver

driver = webdriver.Chrome(executable_path="D:\chromedriver.exe")
#url = 'https://www.dcrustedp.in/show_chart.php'
driver.get('https://www.dcrustedp.in/show_chart.php')

rows = 2
cols = 5

for r in range(5,rows+1):
    for c in range(6,cols+1):
        value = driver.find_element_by_xpath("/html/body/center/table/tbody/tr["+str(r)+"]/td["+str(c)+"]").text
        print(value)

` This is my code. I want to extract result date of B.Tech - Computer Science and Engineering 5th Semester. It is in the first row of table. The date is 24-02-2020. I want to print the date from that particular cell only.

According to the your for loop of 'r', it starts from 5 and finishes on 3 (rows+1). Also same problem in 'c' loop as starts from 6 and finishes on 6 (cols+1). You need to change these intervals (rows+1, 6) and (cols+1,7). — Orhan Solak
– Orhan Solak, Commented Feb 26, 2020 at 23:45
find by xpath is actually a method from selenium. However, there is a library etree which can provide a similar functionality. You can refer to this link. Hope this helps. stackoverflow.com/questions/11465555/… — Prakhar Jhudele
– Prakhar Jhudele, Commented Feb 28, 2020 at 6:04

Prakhar Jhudele · Accepted Answer · 2020-03-05 04:42:06Z

1

The below code works-:

from selenium import webdriver
from bs4 import BeautifulSoup
import time
webpage = 'https://www.dcrustedp.in/show_chart.php'
driver = webdriver.Chrome(executable_path='Your/path/to/chromedriver.exe') 
driver.get(webpage)
time.sleep(15)
html = driver.page_source

soup = BeautifulSoup(html, "html.parser")

pagehits=driver.find_element_by_xpath("/html/body/center/table/tbody/tr[3]/td[5]")
print(pagehits.text)

driver.quit()

Without Selenium, we can use requests library to fetch the table and then respective element

import requests
import pandas as pd
url = 'https://www.dcrustedp.in/show_chart.php'
html = requests.get(url, verify=False).content
df_list = pd.read_html(html)
df = df_list[-1]
print(df.iat[0,4])

edited Mar 5, 2020 at 4:42

answered Feb 27, 2020 at 5:41

Prakhar Jhudele

9651 gold badge7 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Gaurav Sekhri Over a year ago

Thank you Sir ! The code worked perfectly as per my requirement. Also, can you please help me to find a basic-simple code to just print the text from the pre-defined xpath. I don't want the chrome browser to open.

Prakhar Jhudele Over a year ago

@GauravSekhri find by xpath is actually a method from selenium. However, there is a library etree which can provide a similar functionality. You can refer to this link. Hope this helps. stackoverflow.com/questions/11465555/…

Gaurav Sekhri Over a year ago

Can you please make some changes in your code so that the web browser doesn't open each time I run the code?

Prakhar Jhudele Over a year ago

@GauravSekhri I have made an edit to the original answer. Also please click upvote if the changes work for you!

Gaurav Sekhri Over a year ago

Thanks a lot for helping me. Also, when I try to upvote your answer, a pop-up is displayed ("Thanks for the feedback! Votes cast by those with less than 15 reputation are recorded, but do not change the publicly displayed post score.")

|

undetected Selenium · Accepted Answer · 2020-02-27 12:20:25Z

0

To extract the result date of 5th Semester for any of the Prg. Title, you have to induce WebDriverWait for the visibility_of_element_located() and you can use the following Locator Strategy:

xpath:

driver.get('https://www.dcrustedp.in/show_chart.php')
prg_title = "B.Tech - Computer Science and Engineering"
# prg_title = "B.Tech - Electrical Engineering"
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//td[contains(., '"+prg_title+"')]//following-sibling::td[3]"))).get_attribute("innerHTML"))

Console Output:
```
24-02-2020
```

answered Feb 27, 2020 at 12:20

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Collectives™ on Stack Overflow

How to find text of a specific cell from a html table (on a url) using python?

2 Answers 2

7 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related