Node async control flow

Question

I am trying to work with twitter data in node and am running into some road blocks that I think should be related to node style coding. The block of code is meant to grab the tweets, check if the text is in mongo and if not insert it.

The first stumbling block I find is that on trying to print out i to the console it will always iterate through each i before it starts iterating through the cursor. I think if I could clear that up it may help me going forward. Is this enough info to help me out?

What I have is:

T.get('statuses/user_timeline', options , function(err, data) {

     var db = MongoClient.connect('mongodb://127.0.0.1:27017/test', function(err, db) {
        if(err)
            throw err;
        console.log("connected to the mongoDB !");
        myCollection = db.collection('test_collection2');

        for (var i = 0; i < data.length ; i++) {
            //let's wrap this in a loop
            docu = data[i];
            //console.dir(data);
           console.dir(i);
           var cursor = myCollection.find({text : data[i].text}).limit(1);
           cursor.each(function(err, doc) {
              if (doc != null) {
                    console.dir('doc is not null');
                    console.dir(doc.text);
              } else {
                    console.dir('doc is null - inserting');
                    myCollection.insert(docu, function(err, records){
                       console.log("Record added as "+records.text);
                       });
              }
           })
        }
    });
})

What is myCollection.find(...).limit(...) supposed to return ? Is it really a cursor, or a promise resolved when the request is done ? — phtrivier
– phtrivier, Commented Jul 17, 2015 at 13:58
A cursor. Though, even if I can't test now, it's likely .find will return before cursor could be used for iteration. imho, the best practing would be to use ...find({...}, function (err, res) { ... }).limit(1); — Giuseppe Crinò
– Giuseppe Crinò, Commented Jul 17, 2015 at 14:02
The doc seems to suggest using cursor.toArray(function(err, doc) { ...})) mongodb.github.io/node-mongodb-native/api-generated/… , but I don't know how the API behaves, so just my 2 cents... — phtrivier
– phtrivier, Commented Jul 17, 2015 at 14:07

Sven Delueg · Accepted Answer · 2015-07-17 14:14:03Z

1

The problem is just because Javascript is async. The loop is finished before the find function in Mongo gives you a return value.

I would do the Following, or something similar - just to explaine the concept:

T.get('statuses/user_timeline', options , function(err, data) {

 var db = MongoClient.connect('mongodb://127.0.0.1:27017/test', function(err, db) {
    if(err)
        throw err;
    console.log("connected to the mongoDB !");
    myCollection = db.collection('test_collection2');

    var myfunction = function(correct_i,docu){
      var cursor = myCollection.find({text : data[correct_i].text}).limit(1);
       cursor.each(function(err, doc) {
          if (doc != null) {
                console.dir('doc is not null');
                console.dir(doc.text);
          } else {
                console.dir('doc is null - inserting');
                myCollection.insert(docu, function(err, records){
                   console.log("Record added as "+records.text);
                   });
          }
       })
    };

    for (var i = 0; i < data.length ; i++) {
        //let's wrap this in a loop
        docu = data[i];
        //console.dir(data);
       console.dir(i);
       myfunction(i,docu);

    }
    });
})

This way each lookup/find to Mongo will have the correct i. because the function gets it as an parameter

edited Jul 17, 2015 at 14:14

answered Jul 17, 2015 at 13:57

Sven Delueg

1,00111 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Giuseppe Crinò Over a year ago

You can pass a callback to .find, mongodb.github.io/node-mongodb-native/api-generated/…

Sven Delueg Over a year ago

the comment of @giuscri might be the cleaner solution

Collectives™ on Stack Overflow

Node async control flow

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related