Using Selenium with Python to parse table data

Question

<table class="table table-striped">
    <thead>
    <tr class="reactable-column-header">
        <th class="reactable-th-status reactable-header-sortable " role="button" tabindex="0"><strong></strong></th>
        <th class="reactable-th-question_id reactable-header-sortable reactable-header-sort-asc" role="button"
            tabindex="0"><strong>#</strong></th>
        <th class="reactable-th-question_title reactable-header-sortable " role="button" tabindex="0">
            <strong>Title</strong></th>
        <th class="reactable-th-editorial reactable-header-sortable " role="button" tabindex="0">
            <strong>Solution</strong></th>
        <th class="reactable-th-acceptance reactable-header-sortable " role="button" tabindex="0">
            <strong>Acceptance</strong></th>
        <th class="reactable-th-difficulty reactable-header-sortable " role="button" tabindex="0">
            <strong>Difficulty</strong></th>
        <th class="reactable-th-frequency reactable-header-sortable " role="button" tabindex="0"><strong>Frequency
            &nbsp;<span id="frequency-tooltip" class="fa fa-lock" data-toggle="tooltip" data-placement="top" title=""
                        data-original-title="Only premium members can see the frequency"></span></strong></th>
    </tr>
    </thead>
    <tbody class="reactable-data">
    <tr>
        <td label="[object Object]"></td>
        <td label="[object Object]">1</td>
        <td value="Two Sum" label="[object Object]">
            <div><a href="/problems/two-sum">Two Sum</a>&nbsp;&nbsp;&nbsp;&nbsp;</div>
        </td>
        <td label="[object Object]"><a href="/articles/two-sum"><i class="fa fa-file-text"></i></a></td>
        <td value="44.23248982536708" label="[object Object]">44.2%</td>
        <td value="[object Object]" label="[object Object]"><span class="label label-success round">Easy</span></td>
        <td label="[object Object]"></td>
    </tr>
    <tr>
        <td label="[object Object]"></td>
        <td label="[object Object]">2</td>
        <td value="Add Two Numbers" label="[object Object]">
            <div><a href="/problems/add-two-numbers">Add Two Numbers</a>&nbsp;&nbsp;&nbsp;&nbsp;</div>
        </td>
        <td label="[object Object]"><a href="/articles/add-two-numbers"><i class="fa fa-file-text"></i></a></td>
        <td value="31.20978757805531" label="[object Object]">31.2%</td>
        <td value="[object Object]" label="[object Object]"><span class="label label-warning round">Medium</span></td>
        <td label="[object Object]"></td>
    </tr>
    <tr>
        <td label="[object Object]"></td>
    </tbody>
</table>

The HTML above represents the two rows shown in the image. I want to iterate through the table row and print out the Title (Two Sum and Add Two Numbers) using Selenium with Python.

However, the table structure is too complicated and I am not sure how to make a generalized function that might work for a bigger table with more rows.

Any help?

KunduK · Accepted Answer · 2019-06-15 07:24:00Z

If you use selenium and following xpath it will return all cells under rows from table body.

//table[@class='table table-striped']/tbody[@class='reactable-data']//tr//td

However you need to find out the cell index to find particular text or particular tag inside cell.

(Two Sum Add Two Numbers)

in this case your xpath should be

//table[@class='table table-striped']/tbody[@class='reactable-data']//tr//td[3]//a

To handle dynamic element it always good to induce WebdriverWait

Here is your complete code.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver=webdriver.Chrome(executable_path='path/to/chromedriver')
driver.get("url")
elements=WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.XPATH,"  //table[@class='table table-striped']/tbody[@class='reactable-data']//tr//td[3]//a")))
for ele in elements:
    print(ele.text)

Output will be printed on console.

Two Sum
Add Two Numbers

Collectives™ on Stack Overflow

Using Selenium with Python to parse table data

1 Answer 1

Output will be printed on console.

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Output will be printed on console.

Comments

Your Answer

Sign up or log in

Post as a guest

Related