How to fix Selenium Webdriver exception ElementNotVisibleException?

Question

I am trying to parse Upwork with Selenium and getting exception

Exception in thread "main" org.openqa.selenium.ElementNotVisibleException:

And I don't uderstand why. I try click on h2 element that contains link, on link itself, but get still same exception. I check styles of this element in development tools ant this element are visible.

I try use ExpectedConditions and wait until elements became visible, but get timeout exception. What did I do wrong and how I can click on this link?

Of course i can just use uri and open it, but i want solve problem with click.

System.setProperty("webdriver.gecko.driver", "/usr/local/bin/geckodriver");
WebDriver driver = new FirefoxDriver();
driver.get("https://www.upwork.com/o/jobs/browse/");
List<WebElement> titles = driver.findElements(By.className("job-title"));
for (WebElement title: titles) {
    System.out.println("text: " + title.getAttribute("innerText"));
    System.out.println("tag: " + title.getTagName());
    title.click();
    driver.navigate().back();
}

Gaurang Shah · Accepted Answer · 2017-03-07 11:47:09Z

1

I am not sure what exactly you are trying to do, however if you are trying to scrape the website, let me tell you this approach might not work. These websites are equipped with feature to identify bots, later or sooner it will identify your code and you will given captcha to verify.

But let's talk about your code, it's failing because the element is not visible. it's simple, could be because the visibility flag is false or size is not available or other reasons. simple fix is get the href element and your issue is solved.

Just add the TODO code and this code will be running.

    WebDriver driver = new FirefoxDriver();
    driver.get("https://www.upwork.com/o/jobs/browse/");
    List<WebElement> titles = driver.findElements(By.xpath("//h2/a"));
    for (int i=0; i<titles.size(); i++) {
        //TODO add a code to wait until first page loaded

        //To handle StaleElementReferenceException
        WebElement title = driver.findElements(By.xpath("//h2/a")).get(i);
        System.out.println("text: " + title.getAttribute("innerText"));
        System.out.println("tag: " + title.getTagName());
        title.click();
        driver.navigate().back();
    }

answered Mar 7, 2017 at 11:47

Gaurang Shah

13.1k15 gold badges96 silver badges164 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Hayate Over a year ago

I try just collect some statistics about most required jobs and most relevant for me.

Gaurang Shah Over a year ago

@Hayate if it's working that good, what I am saying is if you will run in more frequently the captch will appear.

Hayate Over a year ago

I understand, i add sound notification for case when captch appear, so i solve captch manualy when in required.

Collectives™ on Stack Overflow

How to fix Selenium Webdriver exception ElementNotVisibleException?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related