Memory management is not something that most PHP developers ever need to think about. I'm running into an issue where my command line script is running out of memory. It performs multiple iterations over a large array of objects, making multiple database requests per iteration. I'm sure that increasing the memory ceiling may be a short term fix, but I don't think it's an appropriate long-term solution. What should I be doing to make sure that my script is not using too much memory, and using memory efficiently?
4
-
1Essentially - clean up after each loop. Maybe show some code?Pekka– Pekka2011-09-22 15:32:22 +00:00Commented Sep 22, 2011 at 15:32
-
You're not using anonymous functions in this loop, are you? I've ran into that before...Brad– Brad2011-09-22 15:33:19 +00:00Commented Sep 22, 2011 at 15:33
-
Perhaps, since you're using the CLI anyway, you could consider switching language. There are other languages out there that not only have a smaller memory footprint (which could be debated to death), but give you more control over the memory usage.TJHeuvel– TJHeuvel2011-09-22 15:33:38 +00:00Commented Sep 22, 2011 at 15:33
-
I can't really switch languages on this project, but thanks for the suggestion.Andrew– Andrew2011-09-22 16:11:21 +00:00Commented Sep 22, 2011 at 16:11
Add a comment
|
3 Answers
The golden rule
The number one thing to do when you encounter (or expect to encounter) memory pressure is: do not read massive amounts of data in memory at once if you intend to process them sequentially.
Examples:
- Do not fetch a large result set in memory as an array; instead, fetch each row in turn and process it before fetching the next
- Do not read large text files in memory (e.g. with
file); instead, read one line at a time
This is not always the most convenient thing in PHP (arrays don't cut it, and there is a lot of code that only works on arrays), but in recent versions and especially after the introduction of generators it's easier than ever to stream your data instead of chunking it.
Following this practice religiously will "automatically" take care of other things for you as well:
- There is no longer any need to clean up resources with a big memory footprint by closing them and losing all references to them on purpose, because there will be no such resources to begin with
- There is no longer a need to unset large variables after you are done with them, because there will be no such variables as well
Other things to do
- Be careful of creating closures inside loops; this should be easy to do, as creating such inside loops is a bad code smell. You can always lift the closure upwards and give it more parameters.
- When expecting massive input, design your program and pick algorithms accordingly. For example, you can mergesort any amount of text files of any size using a constant amount of memory.
Comments
You could try profiling it puting some calls to memory_get_usage(), to look for the place where it's peaking.
Of course, knowing what the code really does you'll have more information to reduce its memory usage.
Comments
When you compute your large array of objects, try to not compute it all at once. Walk in steps and process elements as you walk then free memory and take next elements.
It will take more time, but you can manage the amount of memory you use.
