NodeJS MongDB query

Question

I'm building a webapp with NodeJS and a MongoDB database. Currently the following is frustrating me - I am lacking in understanding the callback/(a)sync element and still learning this so I assume that it has something to do with this aspect.

I have two main modules:

DB.js export - this function works when called by itself:

DB.GetSuggestions: function(searchTerm) {
    return MongoClient.connect(url).then(function(db) {
        var collection = db.collection("AccentPairs");
        collection.updateMany(
            {Unstressed: searchTerm},
            {$inc: {QueryCount: 1}}
        );
        var result = collection.aggregate([
            { $match: { Unstressed: searchTerm } },
            { $group: { _id: {"WordID": "$WordID", "WordName": "$WordName", "Unstressed": "$Unstressed", "Stressed": "$Stressed", "TranslationEn": "$TranslationEn"}, Forms: { $push: "$Field" }}}
        ]).sort( { Chosen: -1} ).toArray();
        db.close();
        return result;
    });
}

TextHandler.js:

var DB = require("../DB");

function ImportText(Text) {
  var fullText = Text.trim();
  var TextObject = {AuthorName: "", Title: "", Words: []};
  var currentWord = "";
  var BOS = true;
  var EOS = false;
  let reg = /([а-яА-ЯЁё́]+)([^а-яА-ЯЁё́]*)/g;
  var result;
  while (result = reg.exec(fullText)) {
    BOS = EOS;
    EOS = (result[2].indexOf(".") > -1);
    currentWord = result[1];
    WordObject = {WordText: result[1], WordID: "0", Options: [], Tail: result[2], BOS: BOS, EOS: EOS};
    TextObject.Words.push(WordObject);
  }
  //This is what doesn't work - GetSuggestions within the loop doesn't work.
  TextObject.Words.forEach(function(wd){

    console.log(wd.WordText);
    DB.GetSuggestions(wd.WordText).then(function(suggestions){
      wd.Options = suggestions;
    });
  });
}

I am trying to iterate over the TextObject.Words array and search the database for suggestions using the GetSuggestions function. Calling GetSuggestions within the loop doesn't work, outside of the loop it works.

Error message: <--- Last few GCs --->

29335 ms: Mark-sweep 1386.5 (1440.6) -> 1386.5 (1440.6) MB, 1156.9 / 0.7 ms [allocation failure] [GC in old space req uested]. 30456 ms: Mark-sweep 1386.5 (1440.6) -> 1387.5 (1424.6) MB, 1120.6 / 0.7 ms [last resort gc]. 31576 ms: Mark-sweep 1387.5 (1424.6) -> 1388.4 (1424.6) MB, 1119.4 / 0.7 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000001186EBCFB49 1: /* anonymous /(aka / anonymous */) [C:\Home\CS\TextAnalysis\Ezhik\node_modules\mongodb\lib\url_parser.js:~7] [p c=000003CC6CBB279E] (this=000001186EB04381 ,url=0000004FD766F421 ) 2: arguments adaptor frame: 2->1 3: connect(aka connect) [C:\Home\CS\TextAnalysis\Ezhik\node_modules\mongodb\lib\mongo_client.js:~390] [pc=000...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

What are you expecting to happen? What actually happens? — zero298
– zero298, Commented Oct 9, 2017 at 18:52
No error - it just gets stuck in the loop. Edit: updated post. — pwwolff
– pwwolff, Commented Oct 9, 2017 at 18:57
I didn't copy the full module to save space. ImportText is called from another function in Texthandler.js - a function that opens and reads text files in a specific folder. — pwwolff
– pwwolff, Commented Oct 9, 2017 at 19:36

Dinesh · Accepted Answer · 2017-10-09 19:06:37Z

1

you cannot use forEach in this fashion with Promisified GetSuggestion call. Please use Promises to iterate though TextObject. See the below link(#mistake number 2) to understand how to use Promises for such scenarios.

answered Oct 9, 2017 at 19:06

Dinesh

113 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Dinesh Over a year ago

Oops here is the link - pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html

pwwolff Over a year ago

Great - I thought it might be something of this nature. What I was doing felt wrong. Looks like I need to research promises.

223seneca Over a year ago

Please let us know if this solves the issue for you, as I have another hunch on what might be the problem.

pwwolff Over a year ago

@223seneca I still don't have it fixed... although my grasp of the promise/(a)sync isn't too strong. Have decided to go through some tutorials. Would be happy to hear your hunch :)

223seneca · Accepted Answer · 2017-10-11 01:54:33Z

Since you are receiving a "JavaScript heap out of memory" error, and since the relevant function works when called outside a loop, you are getting stuck in an infinite loop somehow. This is relatively easy to do with while loops, which you use in TextHandler.js.

I suspect your issue is one or more of the following:

DB.GetSuggestions isn't importing into TextHandler.js properly. Can you confirm that the function works within TextHandler.js when it is not inside an iterative loop?
The while loop should use == or === (try both) for comparison in its condition, not =.
In ImportText you create a new undefined variable result and in the next line use it in a comparison condition for a while loop. You probably don't intend for result to be undefined here, and it could be causing your infinite loop (assuming reg.exec(fullText)) is otherwise working correctly).

Let me know if these steps help.

Actung · Accepted Answer · 2017-10-11 12:25:50Z

1

 TextObject.Words.forEach(function(wd){ // iteration

    console.log(wd.WordText);
    DB.GetSuggestions(wd.WordText).then(function(suggestions){ // async block
      wd.Options = suggestions;
    });
  });

You are iterating over async block DB.GetSuggestions(wd.WordText) which is not waiting for async block to finish and moves on to next item in iteration. You need to you Promises to handle async functions with ease. You can your Promise.all.

I found answer from "Iterate over async function" for your question to be most relevant

answered Oct 11, 2017 at 12:25

Actung

1,5361 gold badge13 silver badges19 bronze badges

Collectives™ on Stack Overflow

NodeJS MongDB query

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related