simple_html_dom.php memory problems

Question

I am trying to write a crawler using simple_html_dom.php version 1.5 but it seems it leaks memory for reasons unknown. I tried the 1.5 because they claim to have fixed memory leaks help will be appreciated. after 40 repetitions of the loop i get the following message

   Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 8388608 bytes) in C:\work\simple_html_dom.php on line 1078
<?php
/**
* ******************TESTING*************************
*/




include("simple_html_dom.php");


$beginning=0;
$end=35;
$FileName = "c:/results.txt";
$FileHandle = fopen($FileName, 'w') or die("can't open file");

for ($i = $beginning; $i < $end; $i++) {

$url = sprintf('http://imgur.com/gallery/hot/day/page/%d?scrolled',$i);

$html = file_get_html($url);

echo "Day: -".$i."\n";


foreach($html -> find('div[class=posts]') as $element){




    foreach($element -> find('img') as $el)
    {
        $urls = $el-> src;
        $urls1 = str_replace('b.jpg','.jpg',$el->src);
        $urls2 =     str_replace('.jpg','',str_replace('.com/','.com/gallery/',str_replace('http://i.','http://',str_replace('b.jpg','.jpg',$el->src))));

        $title=str_replace('&quot;','"',str_replace('&#039;',"'",stristr($el-> title,'<p>',true)));
        $fil= $urls2.'             '.$urls.'             '.$urls1.'             '.$title."\n";
        fwrite($FileHandle, $fil);

    }
}

$html->clear;
unset($html);
}

fclose($FileHandle);




?>

I would consider that simple_html_dom is an outdated library that has a broken design. You should replace it with something better, there are other, better libraries available. — hakre
– hakre, Commented Nov 26, 2011 at 16:50
Suggested third party alternatives to SimpleHtmlDom that actually use DOM instead of String Parsing: phpQuery, Zend_Dom, QueryPath and FluentDom. — Gordon
– Gordon, Commented Nov 26, 2011 at 16:51
Thanks i will try them in other projects but since this just a part of the whole it would take some time to change the entire code to use the other libraries. — Evan
– Evan, Commented Nov 26, 2011 at 19:15

dev-null-dweller · Accepted Answer · 2011-11-26 17:25:36Z

3

$html->clear;

if this is your actual code then you may want to change it to function call: $html->clear();

If its not the issue, try downgrading to 1.11, clear() worked there pretty well.

answered Nov 26, 2011 at 17:25

dev-null-dweller

29.5k3 gold badges68 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Evan Over a year ago

Thanks using that function plus running the script in pieces with the help of a .bat file seems to do the trick

Community · Accepted Answer · 2017-05-23 11:55:30Z

1

You could increase the memory with

ini_set("memory_limit","LIMIT");

for example to

ini_set("memory_limit","32M");

btw, check out: PHP Simple HTML Dom Memory Issue

edited May 23, 2017 at 11:55

CommunityBot

11 silver badge

answered Nov 26, 2011 at 17:20

Tieme

66.1k22 gold badges111 silver badges159 bronze badges

Collectives™ on Stack Overflow

simple_html_dom.php memory problems

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related