Scrape web page and retrieve javascript variables

Question

I need to scrape a web page that has a javascript array embeded in inline javascript code, such as:

<script>
    var videos = new Array();
    videos[0] = 'http://myvideos.com/video1.mov'; 
    videos[1] = ....
    ....
</script>

What's the easiest way to approach this and end up with a PHP array of these video urls?

Edit: All videos are .mov extension.

I have a few lines using file_get_contents and trying out a few regexp. I'm no good at regex, tho. — Ignacio
– Ignacio, Commented Jan 12, 2012 at 23:17

Eugen Rieck · Accepted Answer · 2012-01-12 23:54:45Z

1

This is a bit more complicated, but it will get only those links, that are really of the form videos[0] = 'http://myvideos.com/video1.mov';

$tmp=str_replace(array("\r","\n"),'',$original,$matches);
$pattern='/\<script\>\s+var\ videos.*?((\s*videos\[\d+\]\ \=\ .http\:\/\/.*?\;\s*?)+)(.*?)\<\/script\>/';
$a=preg_match_all($pattern,$tmp,$matches);
unset($tmp);

if (!$a) die("no matches");

$pattern="/videos\[\d+\]\ \=\ /";
$matches=preg_split($pattern,$matches[1][0]);

$final=array();
while(sizeof($matches)>0) {
  $match=trim(array_shift($matches));
  if ($match=='') continue;
  $final[]=substr($match,1,-2);
}
unset($matches);

print_r($final);

After feedback from the OP here is the simplified version:

$original=file_get_contents($url);
$pattern='/http\:\/\/.*?\.mov/';
$a=preg_match_all($pattern,$original,$matches);
if (!$a) die("no matches");
print_r($matches[0]);

edited Jan 12, 2012 at 23:54

answered Jan 12, 2012 at 23:35

Eugen Rieck

65.6k10 gold badges75 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Ignacio Over a year ago

Thanks, I'll check on this. I think it can be even easier, since all videos are always .mov

Eugen Rieck Over a year ago

So in fact you want all links to .mov files scraped from that page?

Ignacio Over a year ago

Worked like a charm. Thanks a million!

Adrien Schuler · Accepted Answer · 2012-01-12 23:34:13Z

1

You can scrape this by reading the page with a file_get_contents then retrieve the urls with a regex. This is the simplest way i know, especially if you know the file extensions for your videos. Exemple:

<?php
$file = file_get_contents('http://google.com');
$pattern = '/http:\/\/([a-zA-Z0-9\-\.]+\.[fr|com]+)/i';
preg_match_all($pattern, $file, $matches);
var_dump($matches);

edited Jan 12, 2012 at 23:34

answered Jan 12, 2012 at 23:13

Adrien Schuler

2,4251 gold badge22 silver badges32 bronze badges

1 Comment

Ignacio Over a year ago

This was exactly my first approach to it. I guess there's not much alternative, is it?

Collectives™ on Stack Overflow

Scrape web page and retrieve javascript variables

2 Answers 2

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related