0

I want to separate my code in to smaller functions. But had an issue as driver was not available to all functions. So i declared it as a constant (or is there a better way of doing this ?)

but in 3rd function it is failing on line :

Select dropdown_finance_product = new Select(driver.findElement(By.xpath("//select[@id='ResultsNumber']")));

Here is the console message :

Exception in thread "main" java.lang.NullPointerException
    at Scraping.scrapeit.fetch_urls(scrapeit.java:49)
    at Scraping.scrapeit.main(scrapeit.java:24)

Code :

package Scraping;

import java.io.IOException;
import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.Select;
import org.openqa.selenium.support.ui.WebDriverWait;

import tools.Xls_Reader;

public class scrapeit {

    static WebDriver driver;

    public static void main(String[] args) throws IOException {

        start_browser();
        fetch_urls();
        read_excel();
    }

    public static void start_browser() {

        System.setProperty("webdriver.chrome.driver", "C:\\Chrome Driver\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();

        driver.manage().deleteAllCookies();
        driver.get("http://www.example.com/search/items/new/");
        driver.manage().window().maximize();
        WebDriverWait wait = new WebDriverWait(driver, 20);
        wait.until(ExpectedConditions.titleContains("New items));
    }

    public static void fetch_urls() {

        Select dropdown_finance_product = new Select(driver.findElement(By.xpath("//select[@id='ResultsNumber']")));
        dropdown_finance_product.selectByVisibleText("100");
        System.out.println("Selected 100 Items Dropdown");

        // Open Excel and write urls back
        Xls_Reader datatable = new Xls_Reader("C:\\scrape.xlsx");

        List<WebElement> num_items = driver.findElements(
                By.xpath("//a [contains(@href,'http://www.example.com/search/items/new/latest-')] "));

        for (int i = 0; i < num_items.size(); i++) {
            System.out.println("How many items on page = : " + num_items.size() + " counter = " + i);

            String link = num_items.get(i).getAttribute("href");

            datatable.setCellData("New", "URL", i + 2, link);
            System.out.println("URL : " + link);
        }
    }

    public static void read_excel() {

        // Read in url and process...
        Xls_Reader datatable = new Xls_Reader("C:\\scrape.xlsx");
        int r = datatable.getRowCount("URL");
        int c = datatable.getColumnCount("URL");
        System.out.println("num of rows = " + r + " num of cols = " + c);
    }
}
1
  • What's the failure? How is it related to what you've described in the question body? Commented Mar 7, 2018 at 11:00

1 Answer 1

1

The error says it all :

Exception in thread "main" java.lang.NullPointerException

In class scrapeit you have already declared webdriver as an instance of WebDriver as static as follows :

static WebDriver driver;

But then you are again initializing another driver as a new instance of WebDriver as follows :

WebDriver driver = new ChromeDriver();

Hence you see java.lang.NullPointerException

Solution

A quick solution will instead of creating another instance of the WebDriver you need to use the static instance of WebDriver. So you need to remove WebDriver from WebDriver driver = new ChromeDriver(); as follows :

driver = new ChromeDriver();
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.