HtmlUnit is not executing Javascript inside the page

Question

I have been using HtmlUnit to login into my school portal distantly. As I understand, school portals have always problems with logging in. I know that there are other questions about this topic, but if you don't mind, could you please look though my specific example.

Using my code I want to fill out the login form on the main page to then access my personal profile. However, htmlUnit gives me a run-time error and returns back to the main page.

I attached the code I try to execute and errors that I get. Also, I attached output that I get (It is not what I want). (This is what link I am finally trying to get: https://mistar.oakland.k12.mi.us/novi/StudentPortal/Home/PortalMainPage)

public class MistarLogin {

public static void main(String[] args) throws Exception {

    //java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(java.util.logging.Level.OFF);
    //java.util.logging.Logger.getLogger("org.apache.http").setLevel(java.util.logging.Level.OFF);



    final WebClient webClient = new WebClient(BrowserVersion.CHROME);
    int javaScriptLeftNum = webClient.waitForBackgroundJavaScript(3000);



    // Get the first page
    HtmlPage firstPage = (HtmlPage) webClient.getPage("https://mistar.oakland.k12.mi.us/novi/StudentPortal");

    System.out.println(firstPage.asText());
    System.out.println("\n\n\n\n\n\n\n-------------------------------------------------------------------------------\n\n\n\n\n\n");

    // Get the form that we are dealing with and within that form,
    // find the submit button and the field that we want to change.
    HtmlForm form = firstPage.getFormByName("loginform");

    // Enter login and passwd
    //form.getInputByName("districtid").setValueAttribute("");
    form.getInputByName("Pin").setValueAttribute("email");
    form.getInputByName("Password").setValueAttribute("password");

    // Click "Sign In" button/link
     HtmlInput loginButton = form.getInputByValue("Log In");

     HtmlPage testPage = (HtmlPage) loginButton.click();

    // I added the cookie section but this returns a null pointer exception    
    Set<Cookie> cookie = webClient.getCookieManager().getCookies();

    if(cookie != null){

        Iterator<Cookie> i = cookie.iterator();

        while (i.hasNext()) {

            webClient.getCookieManager().addCookie(i.next());

        }

    }

   // final HtmlPage secondPage = loginButton.click();

Then just print out firstPage, testPage (The output is the same)

This is what output I get:

  Student Portal
   District Website

StudentPortal Login User Name:
Password:
Log In

0

Novi Community School District 1

Loading Information...

This is an error I get (the most essential one):

com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError SEVERE: runtimeError: message=[An invalid or illegal selector was specified (selector: ':input[form="loginform"]' error: Invalid selector: *:input[form="loginform"]).] sourceName=[https://mistar.oakland.k12.mi.us/novi/StudentPortal/Scripts/jquery-1.11.2.min.js?m=20150316043710] line=[2] lineSource=[null] lineOffset=[0]

P.S. Sorry for formating, I am not familiar with Stackoverflow fomating rules. P.P.S. If there are any questions, please ask. I will be looking on this question constantly. Thank you, again.

HtmlUnit's biggest weakness is that it uses really old JavaScript features. This ought to work, though. — chrylis -cautiouslyoptimistic-
– chrylis -cautiouslyoptimistic-, Commented Jan 24, 2019 at 23:07

RBRi · Accepted Answer · 2019-01-26 15:37:28Z

have rewritten your code to make it work, hope that helps. Please have a look at the inline comments to understand how to solve your problem. And finally, you need a good knowledge of all the technologies the web is based on to do web automation. Hope your studies will help you with this.

    final String url = "https://mistar.oakland.k12.mi.us/novi/StudentPortal";

    try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_60)) {
        HtmlPage firstPage = webClient.getPage(url);
        // waitForBackgroundJavaScript has to be called after every action
        webClient.waitForBackgroundJavaScript(2000);

        System.out.println(firstPage.asText());
        System.out.println("-------------------------------------------------------------------------------");

        // Get the form that we are dealing with and within that form,
        // find the submit button and the field that we want to change.
        HtmlForm form = firstPage.getFormByName("loginform");

        // Enter login and passwd
        form.getInputByName("Pin").type(XXXX);
        form.getInputByName("Password").type(YYYY);

        // Click "Sign In" button/link
        HtmlInput loginButton = form.getInputByValue("Log In");
        loginButton.click();
        // wait until the js has done the job
        webClient.waitForBackgroundJavaScript(20000);

        // now get the current page content based on the current window; using
        // HtmlPage homePage = (HtmlPage) loginButton.click();
        // does not work because the page content is async replaced by javascript
        HtmlPage homePage = (HtmlPage) webClient.getCurrentWindow().getEnclosedPage();

        System.out.println(homePage.asText());
        System.out.println("-------------------------------------------------------------------------------");

        DomElement table = homePage.getElementById("stuBannerTable");
        for (DomElement tableRow : table.getElementsByTagName("tr")) {
            if (tableRow.asText().contains("Dmitry Kustarnikov") && tableRow.asText().contains("Novi High School (HS)")) {
                System.out.println("row found ");
                for (DomElement tableCell : table.getElementsByTagName("tr")) {
                    if (tableCell.asText().contains("Dmitry Kustarnikov")) {
                        System.out.println("cell found ");

                        tableCell.click();
                        webClient.waitForBackgroundJavaScript(20000);
                        HtmlPage detailsPage = (HtmlPage) webClient.getCurrentWindow().getEnclosedPage();

                        System.out.println(detailsPage.asText());
                        System.out.println("-------------------------------------------------------------------------------");
                        break;
                    }
                }
            }
        }
    }

Collectives™ on Stack Overflow

HtmlUnit is not executing Javascript inside the page

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related