11

This is the case: I have a 2Gb dump file called myDB.sql. It is a dump file that deletes a existing database and creates a new one, with views and triggers. So I have the string myDB_OLD spread for many lines of the code. I would like to change these strings occurrences to myDB_NEW. I could do this easelly using notePad++. But notepad does not opens a 2Gb file. What I did is a PHP code that reads line by line and find and replace the string I want.

This is the code:

$myfile2 = fopen("myDB.sql", "r") or die("Unable to open file!");//Reads the file
while (!feof($myfile2)) {//Pass trough each line
    $str=fgets($myfile2);//Store the line inside a variable
    if (preg_match("/myDB_OLD/",$str)) {//If the string I want to change exists - replace it and conacatenate
        $newStr .= str_replace("myDB_OLD","myDB_NEW",$str);
    } 
    else {//If not concatenate it
        $newStr .=$str;
    }
}//End of while
fclose($myfile2);
//Save the  newStr in a new file named 
$myfile = fopen("newDB.sql", "w") or die("Unable to open file!");
fwrite($myfile,  $newStr);
echo "finished";

This code retrieve each line of the file changes the string, concatenate in variable and creates a new file. It should works but it is not. I dont know why. I am using xdebug to figure out what is the issue, but no luck.

So I change the approach. Instead of save each line in a variable, I save it directly in a file and that works good.

This is the new code:

$myfile = fopen("newDB.sql", "w") or die("Unable to open file!");//Creates a new file "newDB.sql"

$myfile2 = fopen("myDB.sql", "r") or die("Unable to open file!");//Reads the file
while (!feof($myfile2)) {//Pass trough each line
        $str=fgets($myfile2);//Store the line inside a variable
        if (preg_match("/myDB/",$str)) {//If the string I want to change exists - replace it . (Do not concatenate)
            $strRep=str_replace("myDB","myDB_NEW",$str);
        }
        else {
            $strRep =$str;
        }

        fwrite($myfile,  $strRep);// Add the new line to the file "newDB.sql"
}//End of while
fclose($myfile);
fclose($myfile2);

echo "finished";

Ok, I solved my issue but it raises a thought. What is the issue of the first code? I think the issue is the amount of information to be stored in a PHP variable, 2Gb. So, is there a limit in size to a PHP variable to stores value, in this case, a string ? If yes how can I check or change it? Any php.ini variable?

3
  • What error does it give? And what is the memory limit set in your php.ini file? Commented Nov 23, 2015 at 15:29
  • And as you can import a MySQL dump to any database name, why do you even need to do this in the first place? Commented Nov 23, 2015 at 15:30
  • For the first question. There is no error. Xdebugt do not throw any error. It only stops to work inside the loop. For the memory limit I could check it if I new this is the case this is why I didn check it as my first approach before to rewire the code. For the second question. It was a specift situation and I dont know if I change the database name it will change all occurences inside the dump file like creates triggers and etc. I can test it. Do you know it? Commented Nov 23, 2015 at 15:48

2 Answers 2

10

So, is there a limit in size to a Php variable to stores value, in this case, a string ?

Yes. A string can be as large as up to 2GB (2147483647 bytes maximum). You can override this limit by increasing memory_limit directive in php.ini.

From php7 there's not that limitation in a 64 bit system:

Support for strings with length >= 2^31 bytes in 64 bit builds.

Sign up to request clarification or add additional context in comments.

3 Comments

Good one. Hadn't even thought of that. But, as mentioned above - a PHP application using over 2GB is probably worth refactoring :)
What are you talking about @Rob Quist??? a PHP application using over 2Gb? What??? He's got 2gb of DATA. not 2gb worth of PHP stuff... He's writing a php code to copy over a 2gb database... I'm working with a 10Gb database right now, and worked with databases of up to 200gb in the past... Why are you saying I should do some refactoring? I got a ton of data, and thats it. My php code isn't at fault. Nothing is at fault. There is just a lot of users and a lot of manual and automatic data entry... Every single line in that database is useful and necessary...
I mean if you're code is using 2GB RAM at a single moment, its worth refactoring. Wasn't talking about data, obviously.
4

The maximum number of bytes a script is allowed to allocate is set in php.ini. Look for memory_limit. This is 16Mb after PHP 5.2.0 as a default!

This can be changed by doing a:

ini_set('memory_limit','24M');

The maximum size of a string is 2 GB in PHP, probably because of adressing limitations, if allowed by memory_limit.

4 Comments

Indeed. Allthough you're probably doing something wrong if you need huge amounts there. The line-by-line method you've done there is a proper solution.
Additionally, this limit can be increased using set_memory_limit().
@RobQuist You are quite right, though sometimes when parsing xls or pdf files the size can grow beyond 16M
@Peter, sorry I did not find set_memory_limit, but rather ini_set('memory_limit','16M');

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.