Skip particular Javascript execution in HTML unit

Question

I have a URL. I want to fetch Page-Source of the URL after executing Java Scripts.

Fetch Page source using HtmlUnit : URL got stuck

Initially I suspected that it is due to system resource and High CPU usage, that the URL is getting stuck.

Then I tried to run it on HTML UNIT 2.9 and 2.11. It got stuck on both while parsing. Refer the above question for HTML UNIT code scrape that is getting stuck.

Now I am suspecting that this might be due to JS Execution going into infinite loop.

I want to check what JS files are causing problem and remove them from execution.

If they are JS for sites like google analytics, twitter etc, I may not need them at all.

So I want to find a way to tell HTML Unit to ignore certain JS file and execute the rest.

Does anybody know how to do that ?

nhahtdh · Accepted Answer · 2013-03-06 10:11:10Z

5

Try this. It worked for me:

class InterceptWebConnection extends FalsifyingWebConnection{
    public InterceptWebConnection(WebClient webClient) throws IllegalArgumentException{
        super(webClient);
    }
    @Override
    public WebResponse getResponse(WebRequest request) throws IOException {
        WebResponse response=super.getResponse(request);
        if(response.getWebRequest().getUrl().toString().endsWith("dom-drag.js")){
            return createWebResponse(response.getWebRequest(), "", "application/javascript", 200, "Ok");
        }
        return super.getResponse(request);
    }
}

then write following while setting up your webClient

new InterceptWebConnection(webClient);

edited Mar 6, 2013 at 10:11

nhahtdh

56.9k15 gold badges131 silver badges164 bronze badges

answered Mar 6, 2013 at 7:09

Kunal Kishore

2352 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Shashank Over a year ago

I also faced the same issue.

Jerokdeep Over a year ago

Hi, my web client is created list this WebClient webClient = new WebClient(); Where should I add this interception?

Collectives™ on Stack Overflow

Skip particular Javascript execution in HTML unit

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related