Why use `Buffer.concat(body).toString();` instead of `Uint8Array/Buffer.toString()`

Question

I'm reading this article about gathering request data and it gives the following example:

var body = [];
request.on('data', function(chunk) {
  body.push(chunk);
}).on('end', function() {
  body = Buffer.concat(body).toString();
  // at this point, `body` has the entire request body stored in it as a string
});

Other tutorials suggest this way:

var total = [];
request.on('data', function(chunk) {
  total += chunk;
}).on('end', function() {
  body = total.toString();
  // at this point, `body` has the entire request body stored in it as a string
});

They seem to be equivalent. Why use more elaborate Buffer.concat(body).toString(); then?

Bergi · Accepted Answer · 2016-10-25 11:19:18Z

9

Why use Buffer.concat(body).toString(); instead of UintArray8.toString()?

Because they're doing totally different things. But that's not your real question, chunk is a Buffer as well not an Uint8Array.

The two ways of gathering request data seem to be equivalent. What's the difference?

The second snippet is absolutely horrible code. Don't use it. First of all, it should have been written like this:

var total = "";
request.on('data', function(chunk) {
  total += chunk.toString();
}).on('end', function() {
  // at this point, `total` has the entire request body stored in it as a string
});

Starting with an array is absolute nonsense if you're doing string concatenation on it, and total.toString() was only necessary for the case that there were no data events. total would better be a string right from the beginning. In chunk.toString(), the explicit method call is unnecessary (omitting it would have led to it being called implicitly), but I wanted to show what happens here.

Now, how is converting the chunk buffers to strings and concatenating them different from collecting the buffers in an array, concatenating them to a big buffer and converting that to a string?

The answer is multiple-byte characters. Depending on the encoding and body text, there might be characters that are represented by multiple bytes. It can happen that those bytes come to lie across the border of two chunks (in subsequent data events). With the code that decodes each chunk separately, you'll get an invalid result in those cases.

answered Oct 25, 2016 at 11:19

Bergi

671k162 gold badges1k silver badges1.5k bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Max Koretskyi Over a year ago

thanks, I think this The answer is multiple-byte characters is what I'm looking for. Can you please provide a simple example?

Bergi Over a year ago

@Maximus

Buffer.concat([new Buffer([0xe2]), new Buffer([0x98, 0xba])]).toString() != "" + new Buffer([0xe2]) + new Buffer([0x98, 0xba])

(like '☺' != '��')

Max Koretskyi Over a year ago

I see, thanks. Isn't it better to append bits from the chunck to the body instead of storing them as multiple arrays?

Bergi Over a year ago

@Maximus body is one array of buffers, exactly what Buffer.concat expects. Continously appending to one big buffer is probably inefficient because of copy-on-grow.

Max Koretskyi Over a year ago

I've asked another question which I think is relevant to the matter explained by you here, can you please take a look?

Collectives™ on Stack Overflow

Why use `Buffer.concat(body).toString();` instead of `Uint8Array/Buffer.toString()`

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related