Node.js: Sending stream data over server response(without pipe)

Question

Instead of sending a read stream using pipe I want to send it by sending multiple parts of the file as chunks. The ultimate goal is going to be doing it over websocket, but I also want to get this down first. I was wondering how to use a streams data, end, close, readable events effectively. Here's my code.

const fs = require('fs');
const http = require('http');
const handler = (req, res) => {
  var stream = fs.createReadStream(req.url);
  stream.on('readable', () => {res.send(stream.read());});
};
const server = http.createServer(handler);
server.listen(80);

The ultimate goal, again, is to do it through websocket instead, but the problem it seems is in the way I handle the stream.

EDIT: I finished and tested the speed difference. I got pipe: 0.3 seconds and My Method: 0.1 seconds. To test copy the javascript from these fiddles: Pipe vs. My Method

and I might want to have a header

Rey
– Rey

2017-05-31 16:27:51 +00:00
Commented May 31, 2017 at 16:27 — Rey
– Rey, Commented May 31, 2017 at 16:27

Brad · Accepted Answer · 2017-05-31 16:27:21Z

4

Instead of sending a read stream using pipe I want to send it by sending multiple parts of the file as chunks.

Why? That's exactly what happens when you pipe the stream too.

The ultimate goal is going to be doing it over websocket

Why? If you only need to send data in one direction, there's no need for WebSocket. The receiving end can use the Fetch API or similar. HTTP is quite the capable protocol. In any case, you can also pipe to a WebSocket stream.

I was wondering how to use a streams data, end, close, readable events

You have the readable down just fine. Now all you need to do is add a handler for end as well. Something like this:

stream.on('end', () => {
  res.end();
});

If you just piped the stream though, this is done for your automatically.

answered May 31, 2017 at 16:27

Brad

164k57 gold badges380 silver badges559 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

peteb Over a year ago

Upvote for discouraging use of WebSocket for data streaming in favor of HTTP Streams. WebSockets should be used for serializable data, HTTP does an excellent job of handling data streams.

Rey Over a year ago

I want to be able to better handler the data as it's sent.

Rey Over a year ago

Pipe does it slower. "The flow of data will be automatically managed so that the destination Writable stream is not overwhelmed by a faster Readable stream." link Also using websocket is a core part of the app I'm building. I need to send more than just file data per request. I need to send authentication too. What's more is the method for request, like http, https, or websocket. I do have it set up to use http too, but it's slower.

Brad Over a year ago

@Rey That's simply not true. Streams in recent versions of Node are some of the most optimized components there are. All of your writes are just going to a buffer anyway, as you can only send so much data via TCP before filling up the window, so no matter what you do you're at the mercy of the network and transport layer. As far as authentication, you should be handling that with request header data like every other authenticated web service out there. Then you can appropriately reject the request if auth fails. Don't re-invent HTTP status codes on top of WS on top of HTTP.

Brad Over a year ago

@Rey Even if you needed WebSocket, you should still authenticate the connection via HTTP request data before escalating to WebSocket.

|

Collectives™ on Stack Overflow

Node.js: Sending stream data over server response(without pipe)

1 Answer 1

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related