Is there any programming libraries available that will parse an HTML document, execute JavaScript and then allow me to navigate the DOM? This needs to be performed server side, not client side. Any language will do, but Java, PHP, or Ruby are preferred.
Add a comment
|
9 Answers
2 Comments
Pepijn
+1 Forgot about that one... On my Mac I'd just use Python's Applescript capability to run the JS straight on Safari though.
machine elf
Links for updated community version: envjs.com and github.com/thatcher/env-js
in java: http://lobobrowser.org/cobra/java-html-parser.jsp
this is a a Javascript-aware, CSS-aware HTML parser
the most important feature in relation to your question: It is Javascript-aware. DOM modifications that occur during parsing will be reflected in the resulting DOM.
1 Comment
Florida
Link off, mirror please.
Java has support for javascript with Rhino, also look at this page for server side javascript solutions: http://en.wikipedia.org/wiki/Server-side_JavaScript
Comments
Comments
PhantomJS does this and can be used with any server side language. See some integration modules below for NodeJS and PHP
NodeJS
https://npmjs.org/package/node-phantom
https://github.com/sgentle/phantomjs-node
PHP
Comments
Comments
There are now several projects that do a really good job of this:
PhantomJS is a headless version of WebKit, and there are some helpful wrappers such as CasperJS.
Zombie.js which is a wrapper over jsdom written in Javascript (Node.js).
You need to write JavaScript code to interact with both of these projects. I like Zombie.js better so far, since it is easier to set up, and you can use any Node.js/npm modules in your code.
Comments
node.js ?
Node can run any javascript file in its console. I would try node first & see if it can do what you want as it likely has the largest user base & documentation.
