Parsing JSON url with NODE.JS and keeping it up to date with SOCKET.IO

Question

I am very new to node.js and socket.io and I am trying to figure out how to read a JSON array from an external url, then parse it and display on the main page. Then I believe I use socket.io to keep that connection open and keep the JSON object up to date when a change occurs.

This is what I have so far for node.js.

var http = require("http");
var fs = require('fs');
var options = 'http://api.trakt.tv/user/watching.json/APIKEY/USERNAME';

var server = http.createServer(function(request, response){ 
  console.log('Connection');

http.get(options, function(res){
var data = '';

res.on('data', function (chunk){
    data += chunk;
});

res.on('end',function(){
    var obj = JSON.parse(data);
    console.log( obj );
})

});
  response.end(); 
}); 

server.listen(8888);

When I connect to localhost:8888 I see the console show up with "connection" and then the console logs the contents of the JSON object. This is as far as I have managed to get. I would appreciate and help and pointers on how to get that JSON object displayed and styled on my index page and keep it up to date

TIA Mark

Before I fire off an answer, let me make sure I understand you: You have the code getting your JSON object, but now you want Socket.IO to accept incoming connections to retrieve that object? — Matt Holmes
– Matt Holmes, Commented May 30, 2013 at 21:07
Hi. To be honest I am not entirely sure I need socket.io. Like I said and very new to it. I just want to get the contents of that JSON file and then present that on a page that will update as soon as the JSON file updates its own contents. — Soddengecko
– Soddengecko, Commented May 30, 2013 at 21:27
Well, if you wanted to do it as true push/pull, you'd need Socket.IO or some other WebSocket implementation. If you're willing to let it be a little less "real time", and just have your final client page do a pull every half second or second, it's much easier to coordinate. — Matt Holmes
– Matt Holmes, Commented May 30, 2013 at 21:30
I went through using long polling and it was not the result I wanted. I will also be adding server monitoring to this little app so I would like everything in real time if it is possible. — Soddengecko
– Soddengecko, Commented May 30, 2013 at 21:40

Matt Holmes · Accepted Answer · 2013-05-30 22:37:47Z

2

Okay, now that I understand the problem, here is my answer. It's going to be a bit advice laden, as there is no true "right way" to do what you want. All we can be assured if is that yes, you are going to want to use WebSockets (or in this case socket.io, which can emulate websockets on older browsers).

Your current pull code is probably fine, though you're going to want to tweak that code to run on a timeout so that the latest JSON is pulled every so often. In addition, we want to keep the various moving parts of this seperate: Reading from the API/writing the cache, listening to the cache, and then feeding the cache out to connected clients:

var http = require("http");
var fs = require('fs');
var url = 'http://api.trakt.tv/user/watching.json/APIKEY/USERNAME';
var cacheFile = '/path/to/cachefile';
var connectedSockets = [];

function fetchJson() {
    http.get(url, function(res) {
        body = '';

        res.on('data', function(data) {
            body += data;
        });

        res.on('end', function() {
            fs.writeFileSync(cacheFile, body);
            setTimeout(fetchJson, 1000); // Fetch it again in a second
        });
    })
}

fetchJson(); // Start fetching to our JSON cache

// Start watching our cache file
fs.watch(cacheFile, function(event, filename) {
    if(event == 'change') {
        fs.readFile(cacheFile, function(data) {
            connectedSockets.forEach(function(socket) {
                socket.emit('data', JSON.parse(data));
            });
        });
    }
});

// Now setup our socket server
var io = require('socket.io').listen(8888);
io.sockets.on('connection', function(socket) {
    connectedSockets.push(socket);
});

I don't handle disconnected here (you'll want to remove disconnected or err'ed sockets from the connectedSockets list), and I didn't actually run this...but it should give you an idea of where to head.

On the client, it should be a matter of simply:

var socket = io.connect('http://localhost:8888');
socket.on('data', function (data) {
    // Data will contain your JSON object, do your DOM manip here
});

answered May 30, 2013 at 22:37

Matt Holmes

1,0651 gold badge8 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Soddengecko Over a year ago

Hi Matt. Thank you very much for the code. I tested the code (changing the cache file location) and have a few issues. Since it is using socket.io, I wonder if that should actually be called for in the server.js file? My index.php page has the code you wrote but I added a console.log to output the cache file. This does not work, I get a 404 error on the socket.io.js file and an "Uncaught ReferenceError: io is not defined" Am I doing something wrong here or am I missing something? Cheers.

Matt Holmes Over a year ago

Sounds like your not including the socket.io client library in your PHP. There is a socket.io client JS (the name escapes me) that you'll need to serve up in your front end.

Soddengecko Over a year ago

I think I am getting somewhere now. It grabs the data and writes it to the file. I have socket.io called and the index.php is linking to the file correctly. Sadly I am not getting the data to show up but thanks to you I am on the right path. Cheers

Collectives™ on Stack Overflow

Parsing JSON url with NODE.JS and keeping it up to date with SOCKET.IO

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related