3

Is there a node / npm module which would scrape the contents of a web page or a given URL and execute and evaluate it's JavaScript as a real-life web browser, including XMLHttpRequest object and alike?

I thought that scraperjs was the solution, but sadly it isn't.

All help very appreciated!

Improve this question
2
  • 3
    PhantomJS? I think the term you may be looking for is a headless browser, of which Phantom JS is one. Commented Oct 11, 2016 at 7:06
  • That's the one! Thank you <3 Commented Oct 11, 2016 at 7:07

1 Answer 1

Reset to default
7

Headless browsers based on rendering engine:

Puppeteer is the official automation tool for Chrome. It only supports Chrome/Chromium browsers, (with experimental support for Firefox).

Playwright is an alternative to Puppeteer, with support for more browsers, written by the original creators of puppeteer, who jumped ship from Google to Microsoft. It supports Chrome, Firefox and Safari browsers

Previous versions of my answer, included PhantomJS, SlimerJS,CasperJS and Nightmare all of which are no longer maintaned. For modern browsers, just use one of the above

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

What if I don't want to be headless?
Hi, I have updated my answer, since it was outdated. With Puppeteer, you don't have to be headless. It can be configured to launch a regular window.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.