Stream objects from MongoDB cursor into nodejs HTTP response

Question

NOTE: I don't believe this question is a duplicate of this similar question because it is more specific.

I'm attempting to retrieve multiple objects from Mongo with the nodejs-mongodb-driver and write the objects to an HTTP response as JSON. The objects should be in the form of an array, but i don't want to call toArray() on the cursor because of the memory overhead and I try to avoid large JSON.stringify calls whenever possible.

var response = ... // an http response
collection.find().stream(JSON.stringify).pipe(response); // causes a malformed JSON string

The object in the browser appears as follow.

{"obj", "obj"}{"obj", "obj"} // clearly malformed

Is there an efficient way to do this?

Community · Accepted Answer · 2017-05-23 12:33:41Z

2

I will explain the code you wrote so that you understand why it returns malformed JSON and why you probably need toArray() or the JSONStream libary from the answer you posted.

First collection.find() returns a Cursor object. At that point no data was read. Then, the .stream(JSON.stringify) call returns a readable Stream with the transformation function JSON.stringify. Still no data read.

The .pipe(response) call then reads the entire Stream to the end and for every object it calls the JSON.stringify function. Note that it does really call it for every single object seperately and therefore does not create an array. Instead you get your malformed JSON, object after object.

Now the answer in the question you posted as possible duplicate (Stream from a mongodb cursor to Express response in node.js) would work for you, but it requires an additional libary with a JSONStream. The JSONStream properly handles the CursorStream for JSON output. I don't know if that really reduces the overhead though, but you could try that.

Without an addition libary you will have to use toArray().

edited May 23, 2017 at 12:33

CommunityBot

11 silver badge

answered Apr 13, 2016 at 18:09

x4rf41

5,3672 gold badges24 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Aage Torleif Over a year ago

I know why its malformed. I'm asking for an efficient way to do this. If there's an additional library why don't you suggest one.

x4rf41 Over a year ago

you already posted that answer in your first line of the question. and i still think toArray is efficient. have you done tests? The library is in the answer of the question that you thought was not a duplicate.

Collectives™ on Stack Overflow

Stream objects from MongoDB cursor into nodejs HTTP response

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related