Reading HTML+JavaScript using Java

Question

I can read the HTML contents via http (for example, http://www.foo.com) using Java (with URL and BufferedReader classes). However, a couple of them contain JavaScript. My current app cannot process JavaScript.

What's the best way to read HTML content with JavaScript using Java?

I am open using other languages if it is easier.

Thanks in advance for your help.

UPDATE - Clarification:

A couple HTML contents are generated dynamically using JavaScript. I can see the result (in pure HTML after the JavaScript processing) when viewing them on a browser.

On the other hand, when my Java app retrieves the HTML contents, it says that there is no JavaScript on my app.

Ideally, I want to be able to get the same result as on the browser using my Java app.

Thanks for everyone's response.

What are you doing with the contents returned by a URL after reading from it? Are you evaluating the JavaScript? — Binil Thomas
– Binil Thomas, Commented May 23, 2011 at 19:51
Not sure that I correctly understand your question. You can use SWT Browser widget. It can render HTML and supports JavaScript. — George Suaridze
– George Suaridze, Commented May 23, 2011 at 19:53
@pion What do you mean by "it says that there is no JavaScript on my app". Who says that ? Is it the HTML you got which contains this exact text ? Then you should consider modifying your User Agent String to get the correct content first. HTMLUnit can help you for that and is definitely the best way to go. — Grooveek
– Grooveek, Commented Jan 4, 2012 at 9:42
I have same proplem , can u help me ? stackoverflow.com/questions/20781322/… — ducngm.hn
– ducngm.hn, Commented Dec 26, 2013 at 14:25

Aravindan R · Accepted Answer · 2011-05-23 20:47:52Z

2

HtmlUnit has good JavaScript support and it should (almost) parse the HTML as a web browser.

answered May 23, 2011 at 20:47

Aravindan R

3,0841 gold badge30 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Shan · Accepted Answer · 2011-05-23 20:50:44Z

0

Cobra (http://lobobrowser.org/cobra/getting-started.jsp) will fit your needs

answered May 23, 2011 at 20:50

Shan

5313 gold badges8 silver badges28 bronze badges

Comments

quarks · Accepted Answer · 2011-06-28 18:31:12Z

0

For just HTML parsing you can use HTMLParser (org.htmlparser). However from the way you described your problem, it seems you need a browser, because executing is totally different than just parsing. Cheers.

answered Jun 28, 2011 at 18:31

quarks

35.7k82 gold badges311 silver badges553 bronze badges

Comments

Community · Accepted Answer · 2017-05-23 11:59:25Z

-3

With no doubt you need to use Java html parser:

edited May 23, 2017 at 11:59

CommunityBot

11 silver badge

answered May 23, 2011 at 19:53

r.piesnikowski

2,9711 gold badge28 silver badges33 bronze badges

1 Comment

BalusC Over a year ago

99 of 100 HTML parsers in Java don't understand/execute JavaScript. Please be more specific.

Collectives™ on Stack Overflow

Reading HTML+JavaScript using Java

4 Answers 4

Comments

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related