Scale node.js socket.io@1.. with cluster and socket.io-redis on Heroku

Question

Does anybody know a good solution to scale a node.js - socket.io based app up over multiple cores? I am currently testing the solution presented in the socket.io documentation, to use socket.io over multiple nodes, but without a concrete success.

I have created a playground for this on github: https://github.com/liviuignat/socket.io-clusters which is a bit modified copy of the chat application from the socket.io site. It uses express, cluster, [email protected] and socket.io-redis.

There is currently also an implementation using sticky-session in the branch feature/sticky which seems to work better.

In the end the application needs to be published to Heroku, scaled over multiple dynos.

Initially I tryied doing something like this - to start the server only for the cluster nodes, but I always get the error: failed: Connection closed before receiving a handshake response

if (cluster.isMaster) {    
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  });
} else {
  var server = new Server({
      dirName: __dirname,
      enableSocket: true
    })
    .setupApp()
    .setupRoutes()
    .start();
}

Then I tried starting the server also for master nodes:

if (cluster.isMaster) {
  var server = new Server({
      dirName: __dirname,
      enableSocket: true
    })
    .setupApp()
    .setupRoutes()
    .start();

  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  });
} else {
  var server = new Server({
      dirName: __dirname,
      enableSocket: true
    })
    .setupApp()
    .setupRoutes()
    .start();
}

I also tried it using both sticky-session and socket.io-redis in the branch feature/sticky, which seems to perform with success, but still does not seems to be a good solution:

if (cluster.isMaster) {
  sticky(function() {
    var server = new Server({
        dirName: __dirname,
        enableSocket: true
      })
      .setupApp()
      .setupRoutes();
    return server.http;
  }).listen(3000, function() {
    console.log('server started on 3000 port');
  });

  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  });
} else {
  sticky(function() {
    var server = new Server({
        dirName: __dirname,
        enableSocket: true
      })
      .setupApp()
      .setupRoutes();
    return server.http;
  }).listen(3000, function() {
    console.log('server started on 3000 port');
  });
}

I will do more tests for the next days, but, it would help a lot if anybody could come up with some ideas.

Thanks,

Steffen · Accepted Answer · 2014-09-30 15:15:12Z

2

You are probably looking for socket.io-redis. http://socket.io/blog/introducing-socket-io-1-0/ (scroll to 'Scalability')

Here's a shortened example on how to create the scaffolding with socket.io + express:

var cluster = require('cluster');

var express = require('express')
    , app = express()
    , server = require('http').createServer(app);

var
    io = require('socket.io').listen(server)
    var redis = require('socket.io-redis');
    io.adapter(redis({ host: 'localhost', port: 6379 }));



var workers = process.env.WORKERS || require('os').cpus().length;

/**
 * Start cluster.
 */

if (cluster.isMaster) {

  /**
   * Fork process.
   */

  console.log('start cluster with %s workers', workers-1);
  workers--;
  for (var i = 0; i < workers; ++i) {
    var worker = cluster.fork();
    console.log('worker %s started.', worker.process.pid);
  }

  /**
   * Restart process.
   */

  cluster.on('death', function(worker) {
    console.log('worker %s died. restart...', worker.process.pid);
    cluster.fork();
  });


} else {
  server.listen(process.env.PORT || 9010);
}

Redis has pub/sub and all socket.io nodes need to subscribe to redis to get all messages from a channel. This way one process can broadcast a message to a channel (publish) and all other processes receive the messages with minimal latency to broadcast them to their connected clients (subscribe). You can even extend this with redis based sessions.

The cluster module you are referring to is a bit misleading in my opinion. It helps to create invidiual sub-processes as far as I understand the concept, but doesn't 'synchronize' the channels across multiple nodes. If your clients to not need to communicate with others it's fine. If you want to broadcast messages to all connected clients on all nodes you need the redis module.

edited Sep 30, 2014 at 15:15

answered Sep 30, 2014 at 14:42

Steffen

5996 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Liviu Ignat Over a year ago

Hi Steffen, I am using socket.io-redis with no success.

Steffen Over a year ago

I've added an example which is taken from a project I worked on a while back. At that time things were a little different, but I've updated it to use socket.io-redis for socket.io >=1.0

Liviu Ignat Over a year ago

Thanks, it seems that I've managed to make it work with a simple server (see: github.com/liviuignat/chat-example-cluster/blob/master/index.js). I will play with it a little bit more, because in my main project it still has issues.

Liviu Ignat Over a year ago

Hi, I updated the github sample from the link above (github.com/liviuignat/chat-example-cluster) to be able to host it on Heroku and it seems on Heroku it fails. Sometimes it fails also on my local computer. The url from Heroku is: chat-example-cluster.herokuapp.com. So, I am still struggling with it

Collectives™ on Stack Overflow

Scale node.js socket.io@1.. with cluster and socket.io-redis on Heroku

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related