Python selenium - Access html tags with a table

Question

I have this table:-

<table>
<tbody>
    <tr class="stripe">
        <td colspan="3"/>
    </tr>
    <tr>
        <td style="width: 160px;">Field1:</td>
        <td style="width: 250px;">
        <strong>
        <span id="lblSalesExec">item1</span>
        </strong>
        </td>
        <td>
        </td>
    </tr>
    <tr>
        <td>Field2:</td>
        <td>
        <strong>
        <span id="lblLocation">item2</span>
        </strong>
        </td>
        <td/>
    </tr>
    <tr>
    <th colspan="3">
    <h3 style="color: rgb(255, 255, 255);">Other Details</h3>
    </th>
    </tr>
    <tr>
    <td>Field3:</td>
    <td>
    <span id="lblRef">item3</span>
    </td>
    <td/>
    </tr>
    <tr>
    <td>Field4:</td>
    <td>
    <span id="lblCustomerName">item4</span>
    </td>
    <td/>
    </tr>
    <tr>
    <td>Field5:</td>
    <td>
    <span id="lblCurrentAddress">
    item5-1
    <br/>
    item5-2
    <br/>
    item5-3
    <br/>
    item5-4
    </span>
    </td>
    <td/>
    </tr>
    <tr>
    <td>
    <i class="icon-envelope"/>
    Field6:
    </td>
    <td>
    <input name="txtCustomerEmail" type="text" value="item6" id="txtCustomerEmail" style="width: 250px;"/>
    </td>
    <td/>
    </tr>
    <tr>
    <td>
    <i class="icon-phone"/>
    Field7:
    </td>
    <td>
    <input name="txtCustomerTelNo1" type="text" value="item7" id="txtCustomerTelNo1" style="width: 250px;"/>
    </td>
    <td/>
    </tr>
    <tr>
    <td>
    <i class="icon-phone"/>
    Field8:
    </td>
    <td>
    <input name="txtCustomerTelNo2" type="text" id="txtCustomerTelNo2" style="width: 250px;"/>
    </td>
    <td/>
    </tr>
    <tr>
    <td>
    <i class="icon-phone"/>
    Field9:
    </td>
    <td>
    <input name="txtCustomerTelNo3" type="text" id="txtCustomerTelNo3" style="width: 250px;"/>
    </td>
    <td/>
    </tr>
    <tr>
    <td>Field10:</td>
    <td>
    <span id="lblCurrentVehicle">
    item10
    <br/>
    item10-1
    <br/>
    item10-2
    </span>
    </td>
    <td/>
    </tr>


    <tr>
    <th colspan="3">

    </th>
    </tr>
    <tr>

</tbody>
</table>

How to I use Python selenium to parse a table with lots of varying html tags in its rows? See the attached image below with the expected output table.

This is what I have don so far...

ele = driver.find_element_by_class_name("list")
ele_txt = ele.text
spli = re.split('\n', ele_txt)
# spli1 = re.split(':', ele_txt)

spli

Xwris Stoixeia · Accepted Answer · 2017-02-17 10:28:32Z

1

First you want find_elementS not find_element. If I understood your goal well, you are aiming to get the text from the span id's; hence this is where I will focus my xPath expression. So with the below you should be able to find all elements which correspond to html span id in a clever way; meaning without specifying any particular value for the specific span id/html tag. Here is the xpath:

//tr[@span]

Now using find_elements you can return a list of elements matching the Xpath (so, all the span ids); then you extract the text from them.

for ele in driver.find_elements_by_xpath("//tr[@span]"):
    print ele.text

Best of luck!

Update after OP's comment:

Think of your html tags as a tree. You start from the top and work your way down to the attribute that you want. So if your html tag is input (and the attribute you want to grab is type='submit') this translates like following in xpath: //htmlTag[@attribute='value'] --> so we will have --> //input[@type='submit'] You can focus on an element if you load you xPath helper for Chrome and right click on an element and select Inspect; then it focuses on what you need to select :)

edited Feb 17, 2017 at 10:28

answered Feb 15, 2017 at 22:54

Xwris Stoixeia

1,86122 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

Umar Yusuf Over a year ago

Running your solution, I got out this error: InvalidSelectorException

Xwris Stoixeia Over a year ago

Well, the find_elements is for sure correct because of this: selenium-python.readthedocs.io/locating-elements.html so something it does not like on the actual expression. I' improved my answer and soon I'll come with a method for you - not just a solution, bear with me mate :)

Xwris Stoixeia Over a year ago

I highly recommend to download Xpath helper add on for Chrome. This will make you life a hell of a lot easier and will help you to experiment and understand xPath more

Xwris Stoixeia Over a year ago

chrome.google.com/webstore/detail/xpath-helper/…

Xwris Stoixeia Over a year ago

I uploaded an image with xPath helper. On the left you put your xpath and then it highlights the relevant and text and shows what values it has found on the right

|

Collectives™ on Stack Overflow

Python selenium - Access html tags with a table

1 Answer 1

11 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

11 Comments

Your Answer

Sign up or log in

Post as a guest

Related