0

I am trying to download a web-page using nodejs, javascript. However it seems to have a inifinite loop. Why?

var downloadFinished = false;

var downloadPage = function(url,file) {
  rest.get(url).on('complete', function(result) {
  if (result instanceof Error) {
    console.log('Error:', result.message);
  } else {
    fs.writeFileSync(file, result, 'utf8');
    downloadFinished = true;
   }   
  }); 
};

if(require.main == module) {
    downloadPage('http://google.com', 'new.html');
    while(!downloadFinished) {
       // wait till download finished.
    } 
    // do other stuff make sure download is finished to 'new.html'
}
3
  • 1
    this is begging for a callback instead of infinite while looping. Commented Apr 10, 2014 at 22:18
  • @pennstatephil What you mean? Commented Apr 10, 2014 at 22:22
  • pass in a callback function that you can call when the download is complete. Similar to a success callback when using jquery's $.ajax (api.jquery.com/jQuery.ajax) Commented Apr 10, 2014 at 22:24

2 Answers 2

4

Javascript is single-threaded, if you have a loop like:

while(!downloadFinished) {

}

that loop will keep running forever and no other function will run (your .on('complete' callback can't execute until the while loop finishes, because of the single-threaded nature of Javascript, so it won't ever finish since you don't set downloadFinished = true inside that loop or use a break statement).

To work around this, you can do all your other stuff in a callback which you don't call until the download completed:

var downloadPage = function(url, file, callback) {
  rest.get(url).on('complete', function(result) {
    if (result instanceof Error) {
      console.log('Error:', result.message);
    } else {

      /* Don't use writeFileSync, unless you want to block your server,
        from handling any requests at all until the disk IO completes

        fs.writeFileSync(file, result, 'utf8');
        callback();
      */
      fs.writeFile(file, result, 'utf8', callback);
    }   
  }); 
};

if(require.main == module) {
    downloadPage('http://google.com', 'new.html', function after_download(){
        // do other stuff make sure download is finished to 'new.html'
    });
}
Sign up to request clarification or add additional context in comments.

1 Comment

@celebisait You're better off using fs.writeFile instead of fs.writeFileSync as well, so I just changed that in my answer. fs.writeFile will call your callback when it finishes writing the file to the filesystem, and your server is free to handle other requests, since it's not using a synchronous function.
1

When you call that while(!downloadFinished) it is set to false so you are basically doing while(true).

Option 1

You can use a callback instead of a while loop.

var successCallback = function() {
 //do stuff here.
};

var downloadPage = function(url, file, callback) {
  rest.get(url).on('complete', function(result) {
  if (result instanceof Error) {
    console.log('Error:', result.message);
  } else {
    fs.writeFile(file, result, 'utf8', callback);
   }   
  }); 
};

if(require.main == module) {
    downloadPage('http://google.com', 'new.html', successCallback);
}

Option 2

Check out Promises they will really help you here. You could use Bluebird a nice Promise library you can just add to your package dependencies.

3 Comments

I actually am setting downloadFinished=true when it finishes.
You already got my upvote, but note that the OP mentioned node.js. I don't think they're running this in a browser. I would recommend bluebird for option 2 as it is really an awesome Promise library.
@Paulpro Nice i have actually been looking for a nice promise library for node. I have updated my answer to include it!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.