39

I've been doing some HTML scraping in PHP using regular expressions. This works, but the result is finicky and fragile. Has anyone used any packages that provide a more robust solution? A config driven solution would be ideal, but I'm not picky.

Improve this question
1
  • Have a look at this thread - the question goes into a similar direction Commented Aug 29, 2008 at 8:16

7 Answers 7

Reset to default
28

I would recomend PHP Simple HTML DOM Parser after you have scraped the HTML from the page. It supports invalid HTML, and provides a very easy way to handle HTML elements.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Suggested third party alternatives to SimpleHtmlDom that actually use DOM instead of String Parsing: phpQuery, Zend_Dom, QueryPath and FluentDom.
can you give me example to click on any link on a given page?
5

If the page you're scraping is valid X(HT)ML, then any of PHP's built-in XML parsers will do.

I haven't had much success with PHP libraries for scraping. If you're adventurous though, you can try simplehtmldom. I'd recommend Hpricot for Ruby or Beautiful Soup for Python, which are both excellent parsers for HTML.

Improve this answer

1 Comment

If you're going to be parsing particularly sloppy HTML, make sure you don't use BeautifulSoup 3.1.x (use 3.0.x). 3.1.x uses htmllib as its parser, which is much less forgiving than 3.0.x's use of sgmllib.
5

I would also recommend 'Simple HTML DOM Parser.' It is a good option particularly if your familiar with jQuery or JavaScript selectors then you will find yourself at home.

I have even blogged about it in the past.

Improve this answer

Comments

5

I had some fun working with htmlSQL, which is not so much a high end solution, but really simple to work with.

Improve this answer

2 Comments

late comment but I just found your answer via google.. i like it! :)
Does it work for you even now? It does not seem to work for me...
3

Using PHP for HTML scraping, I'd recommend cURL + regexp or cURL + some DOM parsers though I personally use cURL + regexp. If you have a profound taste of regexp, it's actually more accurate sometimes.

Improve this answer

Comments

2

I've had very good with results with the Simple Html DOM Parser mentioned above as well. And then there's the  tidy Extension for PHP as well which works really well too.

Improve this answer

Comments

2

I had to use curl on my host 1and1.

http://www.quickscrape.com/ is what I came up with using the Simple DOM class!

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.