Scrape your own webpage once it's done loading? (Using PHP)

Question

I have a page that's loading another page in an iframe. The page being called does some stuff dynamically as it's called from my site, so I can't really just call it directly, and even if I could it'd be inefficient to call the same page and run that same dynamic code twice.

So what I'd like to do is once my page is done loading, scrape itself, parsing for some certain unique things, and then handle the parsed results. I know how to parse and handle the results of the parsing, however I'm not quite clear on how to do the self-scraping.

Anyone have any suggestions?

TiA

Thomas Havlik · Accepted Answer · 2010-12-25 05:15:52Z

1

Use AJAX to send the page output to a script, then parse it however you want. I don't really know what you mean by "scraping," and you really need to add more tags.

answered Dec 25, 2010 at 5:15

Thomas Havlik

1,3884 gold badges12 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

J. Scott Elblein Over a year ago

Just now getting back to this issue, thanks for the idea, I'll give it a try. =)

Zecc · Accepted Answer · 2010-12-25 05:30:22Z

1

I must say, the whole idea of scraping your output being inefficient (specially when compared to parsing the rendered page) doesn't really sound right to me.

Still, if your problem is getting your output once the page is rendered, you can use the output buffering control functions:

ob_start();
render_your_page();
$rendered_page = ob_get_contents();
ob_end_flush(); // Sends page, turns off output buffering
scrape($rendered_page);

Edit: I hope I understood your question correctly. It's 5:20AM on Christmas day, and I'm ready for bed :)

edited Dec 25, 2010 at 5:30

answered Dec 25, 2010 at 5:17

Zecc

4,35023 silver badges17 bronze badges

1 Comment

J. Scott Elblein Over a year ago

To help clarify, the reason I'm having to do a strange way of scraping my own content is because lets say for example: I load my own webpage w/an iframe in it, in the iframe a different website is being called where some JS is executed and returns it's results in that iFrame. What I need to do is scrape the returned info from that iFrame and store parts of it, hence the scraping, since I don't have direct access otherwise.

Collectives™ on Stack Overflow

Scrape your own webpage once it's done loading? (Using PHP)

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related