Writing async http returns in Nodejs

Question

I realize this quesiton probably gets asked a million times but I'll try anyway. So I am used to Python and Ruby and 'blocking' imperative code blocks and objects and methods and trying to learn this newfangled Node thing for a project I joined. Try as I might, I can't quite wrap my head around async callbacks and passing return values.

Here's my code:

var getAnHttp = function getAnHttp (args, callback) {                   
  var req = http.request(args, callback);                               

  req.on('error', function(errmsg) {                                     
    console.log('problem with request\t' + errmsg.message);              
  });                                                                   

  req.end();                                                            
};                                                                      

var getWebPageBody = function getWebPageBody (res) {                    
  var pageRes = "";                                                     

  res.setEncoding('utf8');                                              
  res.on('data', function(requestBody) {                                
    pageRes = requestBody;                                              
    console.log('[ DEBUG ] BODY:\t' + pageRes);                         
  });                                                                   
  res.on('end', function() {                                            
    return pageRes;                                                     
  });                                                                   
};                                                                      

exports.captureLink = function captureLink(targetLink) {                
  getAnHttp(targetLink, getWebPageBody);                              
};

And actually calling it:

var crawl   = require('../grab_site');

var aGreatSite    = { hostname : 'hotwebcrawlingaction.xxx',
                      port     : 80,
                      path     : '/asyncpron',
                      method   : 'GET'
                    };

var something = crawl.captureLink(aGreatSite);

getWebPageBody will print my output, but I can't get my returns to bubble up through the funcitons. I know I'm calling something in a sync fashion that and some in async but I can't quite sort this out. While I could certainly just put this in one big function I'm trying to do this right, not hack it out.

Anyway sorry for a noob question--I feel like I'm trying to be functional and OOP+imperative at the same time--i see plenty of other examples but I'm trying not to cargo cult or hail mary this one.

What do you want to see printed out?

paulmelnikow
– paulmelnikow

2013-06-04 23:02:33 +00:00
Commented Jun 4, 2013 at 23:02 — paulmelnikow
– paulmelnikow, Commented Jun 4, 2013 at 23:02

Alex Wayne · Accepted Answer · 2013-06-04 23:06:08Z

4

Async functions will never return something that has to fetched with async. getWebPageBody in your case returns undefined, and then later your callbacks happen.

In synchronous programming you return a value. But in async, you provide it as an argument to a callback function.

So instead, make your function accept a callback argument, and simply call it when you are done.

var getWebPageBody = function(res, callback) {                    
  var pageRes = "";                                                     

  res.setEncoding('utf8');                                              
  res.on('data', function(requestBody) {                                
    pageRes = requestBody;                                              
    console.log('[ DEBUG ] BODY:\t' + pageRes);                         
  });                                                                   
  res.on('end', function() {
    callback(pageRes); // invoke callback
  });
};

// and call it
getWebPageBody(res, function(pageRes) {
  // pageRes is now the thing you expect, inside this function.
});

If you invoke a function which triggered async actions, return will never get any values out of that. It's all about callbacks. You fire your own callback, from the callback of something else.

answered Jun 4, 2013 at 23:06

Alex Wayne

188k52 gold badges328 silver badges363 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ispilledthejava Over a year ago

Thanks, that shows me what I was missing, i was still thinking of the callbacks as propagating 'up' like returns. I'm trying to avoid global vars as much as possible.

Collectives™ on Stack Overflow

Writing async http returns in Nodejs

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related