Javascript array performance crash even with smaller chunks

Question

Given is a big (but not huge) array of strings (in numbers 1000-5000 single strings). I want to perform some calculations and other stuff on these strings. Because it always stopped working when dealing with that one big array, I rewrote my function to recursively fetch smaller chunks (currently 50 elements) - I did this using splice because I thought it would be a nice idea to reduce the size of the big array step by step.

After implementing the "chunk"-version, I'm now able to calculate up to about 2000 string-elements (above that my laptop is becoming extremely slow and crashing after a while).

The question: why is it still crashing, even though I'm not processing that huge array but just small chunks successively?

Thanks in advance.

var file = {some-array} // the array of lines
var portionSize = 50; // the size of the chunks

// this function is called recursively 
function convertStart(i) {
   var size = file.length;
   chunk = file.splice(0,portionSize);
   portionConvert(chunk,i);
}

// this function is used for calculating things with the strings
function portionConvert(chunk,istart) {
   for(var i=0;i<portionSize;i++) {
        // doing some string calculation with the smaller chunk here
   }
   istart += 1;
   convertStart(istart);    // recall the function with the next chunk
}

Gabriel Kunkel · Accepted Answer · 2015-12-23 19:49:06Z

2

From my experience the amount of recursion you're doing can "exceed the stack," unless you narrow down the input values, which is why you were able to do more with less. Keep in mind that for every new function call, the state of the function at the call site is saved in your RAM. If you have a computer with little RAM it's going to get clogged up.

If you're a having a processing problem you should switch to a loop version. Loops don't progressively save the state of the function, just the values. Typically, I would leave recursion for smaller jobs like processing a tree-like/object structures or parsing expressions; some situation where it requires processing to "intuitively go deeper" on something. In the case where you just have one long array, I would just process each of the elements with a forEach, which is a for-loop in a handy wrapper:

file.forEach(function(arrayElement) {
  //  doing some string calculation with the chunk (arrayElement) here
});

Take a look at forEach here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach

edited Dec 23, 2015 at 19:49

answered Dec 23, 2015 at 11:05

Gabriel Kunkel

2,7617 gold badges32 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

elasticman Over a year ago

Thanks for the information! I will try to implement a quick test using a loop version.

Gabriel Kunkel Over a year ago

Let me know if it helps.

elasticman Over a year ago

I made some quick tests that helped me to find out, that the recursion itself wasn't the main problem. It was the calculation that happened inside the function. I removed the part where the graphical data was processed and put it outside the recursive function.The loop was more susceptible to crash the system, because of the larger amount of data at one time.

Collectives™ on Stack Overflow

Javascript array performance crash even with smaller chunks

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related