Parsing HTML table data with xpath and selenium in java

Question

I want to take the data and organize it without the tags. It looks something like this

<table class="SpecTable">
    <col width="40%" />
    <col width="60%" />
    <tr>
        <td class="LightRowHead">Optical Zoom:</td>
        <td class="LightRow">15x</td>
    </tr>
    <tr>
        <td class="DarkRowHead">Digital Zoom:</td>
        <td class="DarkRow">6x</td>
    </tr>
    <tr>
        <td class="LightRowHead">Battery Type:</td>
        <td class="LightRow">Alkaline</td>
    </tr>
    <tr>
        <td class="DarkRowHead">Resolution Megapixels:</td>
        <td class="DarkRow">14 MP</td>
    </tr>
</table>

and I want to be able to extract all the strings of information so that I can store in a plaintext file with just this:

Optical Zoom: 15x Digital Zoom: 6x Battery Type: Alkaline Resolution Megapixels: 14 MP

public static void main(String[] args) {

        FirefoxProfile profile = new FirefoxProfile();
        profile.setPreference("general.useragent.override", "some UA string");
        WebDriver driver = new FirefoxDriver(profile);

        String Url = "http://www.walmart.com/ip/Generic-14-MP-X400-BK/19863348";
        driver.get(Url);
        List<WebElement> resultsDiv = driver.findElements(By.xpath("//table[contains (@class,'SpecTable')//td"));

        System.out.println(resultsDiv.size());
        for (int i=0; i<resultsDiv.size(); i++) {
            System.out.println(i+1 + ". " + resultsDiv.get(i).getText());
        }

I am programming in Java with Selenium and I cannot figure out the correct XPath expression for it.

Can someone figure out why I err on this and maybe give me some pointers on how I can parse this data correctly? Im very new to Selenium and XPaths but I need this for work.

Also if anyone has any good sources for me to learn Selenium and XPath fast, those would also be greatly appreciated!

Richard Ev · Accepted Answer · 2013-03-13 17:38:02Z

7

The spec is surprisingly a very good read on XPath.

You might also try CSS selectors.

Anyway, one way to get the data from a table can be as following:

// gets all rows
List<WebElement> rows = driver.findElements(By.xpath("//table[@class='SpecTable']//tr"));
// for every line, store both columns
for (WebElement row : rows) {
    WebElement key = row.findElement(By.XPath("./td[1]"));
    doAnythingWithText(key.getText());
    WebElement val = row.findElement(By.XPath("./td[2]"));
    doAnythingWithText(val.getText());
}

edited Mar 13, 2013 at 17:38

Richard Ev

54.3k61 gold badges196 silver badges282 bronze badges

answered Apr 25, 2012 at 21:43

Petr Janeček

38.5k13 gold badges128 silver badges149 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Aleh Douhi · Accepted Answer · 2012-04-25 21:37:19Z

2

Probably this will suite your needs:

string text = driver.findElement(By.cssSelector("table.SpecTable")).getText();

String text will contain all text nodes from the table with class SpecTable. I prefer using css, because it's supported by IE and faster than xpath. But as for xpath tutorials try this and this.

answered Apr 25, 2012 at 21:37

Aleh Douhi

1,9781 gold badge15 silver badges15 bronze badges

Comments

Nashibukasan · Accepted Answer · 2012-04-25 23:28:45Z

0

As another option you could grab all the cells of the table into one array and access them that way. EG.

ReadOnlyCollection<IWebElement> Cells = driver.FindElements(By.XPath("//table[@class='SpecTable']//tr//td"));

This will get you all the cells in that table as an array which you can then use to access the text iteratively.

string forOutput = Cells[i].Text;

answered Apr 25, 2012 at 23:28

Nashibukasan

2,02824 silver badges37 bronze badges

Comments

florentbr · Accepted Answer · 2012-07-17 01:03:54Z

-1

CSharp method to extract any table in a 2 dimension array:

private string[,] getYourSpecTable(){
    return getArrayBy(By.CssSelector("table.SpecTable tr"), By.CssSelector("td"));
}

private string[,] getArrayBy(By rowsBy, By columnsBy){
    bool init=false;
    int nbRow=0, nbCol=0;
    string[,] ret = null;
    ReadOnlyCollection<OpenQA.Selenium.IWebElement> rows = this.webDriver.FindElements(rowsBy);
    nbRow = rows.Count;
    for(int r=0;r<nbRow;r++) {
        ReadOnlyCollection<OpenQA.Selenium.IWebElement> cols = rows[r].FindElements(columnsBy);
        if(!init) {
            init= true;
            nbCol = cols.Count;
            ret = new string[rows.Count, cols.Count];
        }                
        for(int c=0;c<nbCol;c++) {
            ret[r, c] = cols[c].Text;
        }
    }
    return ret;
}

answered Jul 17, 2012 at 1:03

florentbr

1

1 Comment

Horcrux7 Over a year ago

This is a Java question and not a csharp question.

Collectives™ on Stack Overflow

Parsing HTML table data with xpath and selenium in java

4 Answers 4

Comments

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related