PHP Crawler for a asp.net site

Question

I wanna write a crawler to fetch data. from an asp.net site which uses javascript to do the pagination

You are aware that their T&C state You agree not to reproduce, duplicate, copy, sell, resell or exploit for any commercial purposes, any portion of the Service, use of the Service, or access to the Service ? — Pekka
– Pekka, Commented Oct 31, 2010 at 9:41
Why would you like to get contents from a 3rd party website? You'd better to ask them for permission / the contents before creating a crawler. — Lekensteyn
– Lekensteyn, Commented Oct 31, 2010 at 9:41

Svisstack · Accepted Answer · 2010-10-31 09:41:46Z

2

Use CURL Library to get html code of each page, then you must only parse this pages. For other type of page you must write other parser, do it on a DOM Library then parsers will have few lines for each type of page.

But you must have permission to do this, you must ask the author of agree of what you doing.

answered Oct 31, 2010 at 9:41

Svisstack

16.7k8 gold badges70 silver badges101 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

PHP Crawler for a asp.net site

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related