Multiple embedded loops in NodeJS

Question

I like to execute the following code...but as you can see - it will require 10 billion loops! So i was wondering - what you guys would suggest to make it spin faster?

The reason why - i need to like "brute force" to the best result - is because the most inner method do some complex calculation of some historical data - about 7 million rows from a DB...and the point by doing all this - is to find the best "setting" of the given parameters a-f...which gives the best result...

var a = 0.1;

while(a <= 10) // 100
{
    var b = 0.1;

    while(b <= 10) // 100
    {
        var c = 0.1;

        while(c <= 10) // 100
        {
            var d = 0.1;

            while(d <= 10) // 100
            {
                var e = 1;

                while(e <= 10) // 10
                {
                    var f = 1;

                    while(f <= 10) // 10 10.000.000.000
                    {
                        // Call at method which use all parameters and return the result of a given calculation
                        //doSomeThing(a,b,c,d,e,f);
                        f += 1;
                    }
                    e += 1;
                }
                d += 0.1;
            }
            c += 0.1;
        }
        b += 0.1;
    }
    a += 0.1;
}

if you want to try 10 billions combinations, then you need 10 billion loops, there is no workaround, only to try less combinations (i.e. based on previous doSomething result you know that some of values will not be good or have wider range of results). Also 10 billions loops is computable by personal PC, if the doSomething method is not too difficult to compute and if you only run this from time to time. — libik
– libik, Commented Mar 22, 2016 at 13:05
1. NodeJS is by design async - so a lot of multiple call would be instantiated simultaneously to the inner method ... but it would still just run in a single tread - how do i run the code via like 100 threads simultaneously? Simple to speed up the process? Btw - i need to run this code like all the time - to constantly know what the best "settings" is like... because the historical data also change from one minute to the next... — PabloDK
– PabloDK, Commented Mar 22, 2016 at 13:12
That would be hugely cpu intensive... You could use promises (with a library like Bluebird), but considering your code (especially the call on doSomething(a,b,c,d,e,f)) you'll have to change it substantially. — Cithel
– Cithel, Commented Mar 22, 2016 at 13:35
Not sure if you noticed (probably a copy paste error) but you've got 6 infinite loops there. Switching the increment of a,b,c,d,e,f from inner to outer will fix it. — binarymax
– binarymax, Commented Mar 22, 2016 at 15:27

binarymax · Accepted Answer · 2016-03-22 15:45:24Z

2

Break the loops into smaller chunks and fork processes using the core cluster module, processing each smaller chunk in a fork. The forks will run on separate threads, better leveraging the CPU.

https://nodejs.org/api/cluster.html#cluster_how_it_works

UPDATE. OK, don't use Cluster. Use the threads module instead - it will be much easier. https://www.npmjs.com/package/threads ...

var cores = require('os').cpus().length;
var Pool = require('threads').Pool; 
var pool = new Pool(cores);

var doSomeThing = pool.run(function(data,done){
    var result = data.a * data.b * data.c * data.d * data.e * data.f;
    done(result,data);
})


var a = 0.1;
var b = 0.1;
var c = 0.1;
var d = 0.1;
var e = 0.1;
var f = 0.1;


while(a <= 10) // 100 loops
{
    while(b <= 10) // 100 loops
    {
        while(c <= 10) // 100 loops
        {
            while(d <= 10) // 100 loops
            {
                while(e <= 10) // 10 loops
                {
                    while(f <= 10) // 10 loops
                    {
                        // Call at method which use all parameters and return the result of a given calculation
                        doSomeThing.send({a:a,b:b,c:c,d:d,e:e,f:f});
                        f += 0.1;
                    }
                    e += 0.1;
                }
                d += 0.1;
            }
            c += 0.1;
        }
        b += 1;
    }
    a += 1;
}

pool
  .on('error', function(job, error) {
    console.error('Job errored:', job);
  })
  .on('finished', function() {
    console.log('Everything done, shutting down the thread pool.');
    pool.killAll();
  });

edited Mar 22, 2016 at 15:45

answered Mar 22, 2016 at 13:39

binarymax

3,4353 gold badges22 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

PabloDK Over a year ago

How would i break my loops into smaller parts? Besides from this - i dont need to change my current code - but simply add and use the cluster module in my NodeJS app? And then every thing will scale and perform to the max automatically - by using max numbers of cores/thread?

binarymax Over a year ago

@PabloDK Sorry, its not that easy :) You can't gain the benefit of parallelism without restructuring your code. When I have some time later I will mockup a quick demo.

PabloDK Over a year ago

Thank you so much :-) I'm new to NodeJS...im used to .NET... but it should not be that complicated to make a (NodeJS) app. multi-threaded?

binarymax Over a year ago

Actually, it is hard to make a node.js app multi-threaded! Node is based on an evented IO model, and in general should not be used for CPU intensive applications. When the need does arise, there are ways to do it (cluster being one of them), but they are usually uncomfortable because mixing threads with events is nasty to debug.

PabloDK Over a year ago

IO model or not - more than 1 thread is nasty to debug i think :-D My problem is quite simple though - and i will not need any debugging...i simply want to execute my method/all 10 billion combination - in the fastest possible way... "thats all" ... For fun - i just tested the code above with the 6 loops - but with NO method inside (consuming time) - the NodeJS runtime enviroment is able to complete a single run - of all 10 billions combinations ... in 34 sec at my DELL XPS 13 - i7 laptop... quite impressive - what a pc can handle compared to the human brain ...haha

|

Collectives™ on Stack Overflow

Multiple embedded loops in NodeJS

1 Answer 1

9 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related