682

How do I download a file with Node.js without using third-party libraries?

I don't need anything special. I only want to download a file from a given URL, and then save it to a given directory.

4
  • 132
    "I only want to download a file from a given url, and then save it to a given directory," it seems pretty clear. :) Commented Aug 14, 2012 at 2:26
  • 58
    Joseph is making an incorrect assertion that all node processes are server processes Commented Dec 8, 2013 at 14:36
  • 8
    @lededje What prevents a server process from downloading a file and saving it to a directory on a server? It is prefectly doable. Commented Feb 21, 2020 at 8:51
  • Note that as of Node 18, fetch is built in and no third party libraries or hand-rolled code involving caching data chunks or pipes is necessary anymore. Commented Jan 27, 2024 at 3:15

34 Answers 34

919

As of Node 18, you can use the built-in fetch global, which implements the Fetch API to download data with several methods built in to directly work with the result as plain text, JS-converted-from-JSON, or binary data (as ArrayBuffer).

For older versions of Node, you can create an HTTP GET request and pipe its response into a writable file stream:

const http = require('http'); // or 'https' for https:// URLs
const fs = require('fs');

const file = fs.createWriteStream("file.jpg");
const request = http.get("http://i3.ytimg.com/vi/J---aiyznGQ/mqdefault.jpg", function(response) {
   response.pipe(file);

   // after download completed close filestream
   file.on("finish", () => {
       file.close();
       console.log("Download Completed");
   });
});

If you want to support gathering information on the command line--like specifying a target file or directory, or URL--check out something like Commander.

More detailed explanation in https://sebhastian.com/nodejs-download-file/

Sign up to request clarification or add additional context in comments.

20 Comments

I got the following console output when I ran this script: node.js:201 throw e; // process.nextTick error, or 'error' event on first tick ^ Error: connect ECONNREFUSED at errnoException (net.js:646:11) at Object.afterConnect [as oncomplete] (net.js:637:18) .
Does this code close the file properly when the script ends or would it lose data?
@quantumpotato Take a look at the response you're getting back from your request
This depends upon the req url type if you are requesting https you must use https otherwise it will throw error.
@EthanKeiley why do you say that it isn't closed properly? By default createWriteStream will set autoClose to true and readable.pipe will call end() on the writeable when the readable ends.
|
592

Don't forget to handle errors! The following code is based on Augusto Roman's answer.

var http = require('http');
var fs = require('fs');

var download = function(url, dest, cb) {
  var file = fs.createWriteStream(dest);
  var request = http.get(url, function(response) {
    response.pipe(file);
    file.on('finish', function() {
      file.close(cb);  // close() is async, call cb after close completes.
    });
  }).on('error', function(err) { // Handle errors
    fs.unlink(dest); // Delete the file async. (But we don't check the result)
    if (cb) cb(err.message);
  });
};

13 Comments

@vince-yuan is download() itself pipeable?
@VinceYuan the callback is confusing me. if I now invoke download(), how would I do it? What would I place as the cb argument? I have the download('someURI', '/some/destination', cb) but don't understand what to put in the cb
@Abdul Sounds like you are very new to node.js/javascript. Take a look at this tutorial: tutorialspoint.com/nodejs/nodejs_callbacks_concept.htm It's not complex.
@Abdul maybe it would be good if you share with the rest of the class what you have figured out ?
Is there a way to see the speed of the download? Like can track how many mb/s? Thanks!
|
180

As Michelle Tilley said, but with the appropriate control flow:

var http = require('http');
var fs = require('fs');

var download = function(url, dest, cb) {
  var file = fs.createWriteStream(dest);
  http.get(url, function(response) {
    response.pipe(file);
    file.on('finish', function() {
      file.close(cb);
    });
  });
}

Without waiting for the finish event, naive scripts may end up with an incomplete file.

Edit: Thanks to @Augusto Roman for pointing out that cb should be passed to file.close, not called explicitly.

3 Comments

the callback is confusing me. if I now invoke download(), how would I do it? What would I place as the cb argument? I have the download('someURI', '/some/destination', cb) but don't understand what to put in the cb
@Abdul You specify the callback with a function only if you need to do something when the file has been successfully fetched.
It would be good to check the status code before saving: response.statusCode == 200
94

Speaking of handling errors, it's even better listening to request errors too. I'd even validate by checking response code. Here it's considered success only for 200 response code, but other codes might be good.

const fs = require('fs');
const http = require('http');

const download = (url, dest, cb) => {
    const file = fs.createWriteStream(dest);

    const request = http.get(url, (response) => {
        // check if response is success
        if (response.statusCode !== 200) {
            return cb('Response status was ' + response.statusCode);
        }

        response.pipe(file);
    });

    // close() is async, call cb after close completes
    file.on('finish', () => file.close(cb));

    // check for request error too
    request.on('error', (err) => {
        fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
    });

    file.on('error', (err) => { // Handle errors
        fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
    });
};

Despite the relative simplicity of this code, I would advise to use the request module as it handles many more protocols (hello HTTPS!) which aren't natively supported by http.

That would be done like so:

const fs = require('fs');
const request = require('request');

const download = (url, dest, cb) => {
    const file = fs.createWriteStream(dest);
    const sendReq = request.get(url);
    
    // verify response code
    sendReq.on('response', (response) => {
        if (response.statusCode !== 200) {
            return cb('Response status was ' + response.statusCode);
        }

        sendReq.pipe(file);
    });

    // close() is async, call cb after close completes
    file.on('finish', () => file.close(cb));

    // check for request errors
    sendReq.on('error', (err) => {
        fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
    });

    file.on('error', (err) => { // Handle errors
        fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
    });
};

EDIT:

To make it work with https, change

const http = require('http');

to

const http = require('https');

11 Comments

The request module just works straight for HTTPs. Cool!
@ventura yep, btw, there's also the native https module that now can handle secure connections.
It's more error prone without a doubt. Anyway, in any case where using request module is an option, I'd advise it as it's way higher level and thus, easier and efficient.
@Alex, nope, this is an error message and there's a return. So if response.statusCode !== 200 the cb on finish will never be called.
Thank you for showing example using request module.
|
53

gfxmonk's answer has a very tight data race between the callback and the file.close() completing. file.close() actually takes a callback that is called when the close has completed. Otherwise, immediate uses of the file may fail (very rarely!).

A complete solution is:

var http = require('http');
var fs = require('fs');

var download = function(url, dest, cb) {
  var file = fs.createWriteStream(dest);
  var request = http.get(url, function(response) {
    response.pipe(file);
    file.on('finish', function() {
      file.close(cb);  // close() is async, call cb after close completes.
    });
  });
}

Without waiting for the finish event, naive scripts may end up with an incomplete file. Without scheduling the cb callback via close, you may get a race between accessing the file and the file actually being ready.

5 Comments

What for are you storing request into a variable?
he "stores" it into a variable so it does not become a global variable by default.
@philk how do you know a global variable is created if var request = is removed?
You are right, there is no need to save the request, its not used anyway. That's what you mean?
@philk thank you. Yes, I think that's what polkovnikov.ph meant.
30

Maybe node.js has changed, but it seems there are some problems with the other solutions (using node v8.1.2):

  1. You don't need to call file.close() in the finish event. Per default the fs.createWriteStream is set to autoClose: https://nodejs.org/api/fs.html#fs_fs_createwritestream_path_options
  2. file.close() should be called on error. Maybe this is not needed when the file is deleted (unlink()), but normally it is: https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options
  3. Temp file is not deleted on statusCode !== 200
  4. fs.unlink() without a callback is deprecated (outputs warning)
  5. If dest file exists; it is overridden

Below is a modified solution (using ES6 and promises) which handles these problems.

const http = require("http");
const fs = require("fs");

function download(url, dest) {
    return new Promise((resolve, reject) => {
        const file = fs.createWriteStream(dest, { flags: "wx" });

        const request = http.get(url, response => {
            if (response.statusCode === 200) {
                response.pipe(file);
            } else {
                file.close();
                fs.unlink(dest, () => {}); // Delete temp file
                reject(`Server responded with ${response.statusCode}: ${response.statusMessage}`);
            }
        });

        request.on("error", err => {
            file.close();
            fs.unlink(dest, () => {}); // Delete temp file
            reject(err.message);
        });

        file.on("finish", () => {
            resolve();
        });

        file.on("error", err => {
            file.close();

            if (err.code === "EEXIST") {
                reject("File already exists");
            } else {
                fs.unlink(dest, () => {}); // Delete temp file
                reject(err.message);
            }
        });
    });
}

3 Comments

Two comments on this: 1) it should probably reject Error objects, not strings, 2) fs.unlink will quietly swallow errors which might not necessarily be what you want to do
This works great! And if your URLs use HTTPS, just substitute const https = require("https"); for const http = require("http");
Per the documentation of fs.createWriteStream(): "If autoClose is set to true (default behavior) on 'error' or 'finish' the file descriptor will be closed automatically. " So no need to manually close the file on error.
22

Based on the other answers above and some subtle issues, here is my attempt.

  1. Check the file does not exist before hitting the network by using fs.access.
  2. Only create the fs.createWriteStream if you get a 200 OK status code. This reduces the amount of fs.unlink commands required to tidy up temporary file handles.
  3. Even on a 200 OK we can still possibly reject due to an EEXIST file already exists (imagine another process created the file whilst we were doing network calls).
  4. Recursively call download if you get a 301 Moved Permanently or 302 Found (Moved Temporarily) redirect following the link location provided in the header.
  5. The issue with some of the other answers recursively calling download was that they called resolve(download) instead of download(...).then(() => resolve()) so the Promise would return before the download actually finished. This way the nested chain of promises resolve in the correct order.
  6. It might seem cool to clean up the temp file asynchronously, but I chose to reject only after that completed too so I know that everything start to finish is done when this promise resolves or rejects.
const https = require('https');
const fs = require('fs');

/**
 * Download a resource from `url` to `dest`.
 * @param {string} url - Valid URL to attempt download of resource
 * @param {string} dest - Valid path to save the file.
 * @returns {Promise<void>} - Returns asynchronously when successfully completed download
 */
function download(url, dest) {
  return new Promise((resolve, reject) => {
    // Check file does not exist yet before hitting network
    fs.access(dest, fs.constants.F_OK, (err) => {

        if (err === null) reject('File already exists');

        const request = https.get(url, response => {
            if (response.statusCode === 200) {
       
              const file = fs.createWriteStream(dest, { flags: 'wx' });
              file.on('finish', () => resolve());
              file.on('error', err => {
                file.close();
                if (err.code === 'EEXIST') reject('File already exists');
                else fs.unlink(dest, () => reject(err.message)); // Delete temp file
              });
              response.pipe(file);
            } else if (response.statusCode === 302 || response.statusCode === 301) {
              //Recursively follow redirects, only a 200 will resolve.
              download(response.headers.location, dest).then(() => resolve());
            } else {
              reject(`Server responded with ${response.statusCode}: ${response.statusMessage}`);
            }
          });
      
          request.on('error', err => {
            reject(err.message);
          });
    });
  });
}

4 Comments

You shouldn't waste resources doing the download if the destination file already exists. Do the check first if possible.
@Phil Really good point. I have updated the example with an earlier guard check before falling into the recursive networking calls but left the success 200 file handling code the same. This short circuit guard statement should save some time in this case now.
@JoshPeak what if the file is updated on server.
@y_159 Good question... This solution is only checking the name of the resource. If the file is different on the server then comparing modifed timestamps and hashes of content would be required conditions to invalidate the local cache. Correct cache invalidation is beyond the scope of this question and warrants it's own question and depends how servers implement HTTP ETag protocols.
21

for those who came in search of es6-style promise based way, I guess it would be something like:

var http = require('http');
var fs = require('fs');

function pDownload(url, dest){
  var file = fs.createWriteStream(dest);
  return new Promise((resolve, reject) => {
    var responseSent = false; // flag to make sure that response is sent only once.
    http.get(url, response => {
      response.pipe(file);
      file.on('finish', () =>{
        file.close(() => {
          if(responseSent)  return;
          responseSent = true;
          resolve();
        });
      });
    }).on('error', err => {
        if(responseSent)  return;
        responseSent = true;
        reject(err);
    });
  });
}

//example
pDownload(url, fileLocation)
  .then( ()=> console.log('downloaded file no issues...'))
  .catch( e => console.error('error while downloading', e));

1 Comment

responseSet flag caused, for some reason which I hadn't had the time to investigate, my file to be downloaded incompletely. No errors popped up but the .txt file I was populating had half of the rows that needed to be there. Removing the logic for the flag fixed it. Just wanted to point that out if someone had the issues with the approach. Still, +1
19

Hi,I think you can use child_process module and curl command.

const cp = require('child_process');

let download = async function(uri, filename){
    let command = `curl -o ${filename}  '${uri}'`;
    let result = cp.execSync(command);
};


async function test() {
    await download('http://zhangwenning.top/20181221001417.png', './20181221001417.png')
}

test()

In addition,when you want download large、multiple files,you can use cluster module to use more cpu cores.

2 Comments

This answer is super simple and easy. You can use -L flag at the end to make it follow redirect download (example from github.com).
This is the only answer that worked for me.
17

Solution with timeout, prevent memory leak :

The following code is based on Brandon Tilley's answer :

var http = require('http'),
    fs = require('fs');

var request = http.get("http://example12345.com/yourfile.html", function(response) {
    if (response.statusCode === 200) {
        var file = fs.createWriteStream("copy.html");
        response.pipe(file);
    }
    // Add timeout.
    request.setTimeout(12000, function () {
        request.abort();
    });
});

Don't make file when you get an error, and prefere to use timeout to close your request after X secondes.

3 Comments

this is only a file, has no protocol or server to download from... http.get("http://example.com/yourfile.html",function(){})
Is there a memory leak in this answer: stackoverflow.com/a/22793628/242933?
You can add timeout like I did in http.get. The memory leak is only if the file take too long to be downloaded.
10

Modern version (ES6, Promise, Node 12.x+ ) works for https/http. ALso it supports redirects 302 & 301. I decided do not use 3rd party libraries due to it can be easy done with standard Node.js libs.

// download.js
import fs from 'fs'
import https from 'https'
import http from 'http'
import { basename } from 'path'
import { URL } from 'url'

const TIMEOUT = 10000

function download (url, dest) {
  const uri = new URL(url)
  if (!dest) {
    dest = basename(uri.pathname)
  }
  const pkg = url.toLowerCase().startsWith('https:') ? https : http

  return new Promise((resolve, reject) => {
    const request = pkg.get(uri.href).on('response', (res) => {
      if (res.statusCode === 200) {
        const file = fs.createWriteStream(dest, { flags: 'wx' })
        res
          .on('end', () => {
            file.end()
            // console.log(`${uri.pathname} downloaded to: ${path}`)
            resolve()
          })
          .on('error', (err) => {
            file.destroy()
            fs.unlink(dest, () => reject(err))
          }).pipe(file)
      } else if (res.statusCode === 302 || res.statusCode === 301) {
        // Recursively follow redirects, only a 200 will resolve.
        download(res.headers.location, dest).then(() => resolve())
      } else {
        reject(new Error(`Download request failed, response status: ${res.statusCode} ${res.statusMessage}`))
      }
    })
    request.setTimeout(TIMEOUT, function () {
      request.abort()
      reject(new Error(`Request timeout after ${TIMEOUT / 1000.0}s`))
    })
  })
}

export default download

Kudo to Andrey Tkachenko for his gist which I modified

Include it in another file and use

const download = require('./download.js')
const url = 'https://raw.githubusercontent.com/replace-this-with-your-remote-file'
console.log('Downloading ' + url)

async function run() {
  console.log('Downloading file')
  try {
    await download(url, 'server')
    console.log('Download done')
  } catch (e) {
    console.log('Download failed')
    console.log(e.message)
  }
}

run()

2 Comments

Awesome. Very clean, thank you. What do the flags 'wx' do when you're creating the writeStream?
'wx': Like 'w' but fails if the path exists.
10

Late 2022 edit:

Node v18 and on come with native Fetch API support built right into Node itself. No need for third party libraries, imports, requires, or small hand-crafted shims, just use global fetch the way you're used to from the browser.

(I.e. the second code block, below, does not need the const fetch = require(`./that-code-shown-above.js`); line anymore, fetch already exists globally)

Original answer:

For Node with Promise support, a simple Node shim for (part of) the Fetch API requires only a smattering of extra code, rather than needing to install any special modules:

const http = require(`http`);
const https = require(`https`);

module.exports = function fetch(url) {
  // we're returning a promise, so this function can also be `await`ed
  return new Promise((resolve, reject) => {
    const data = [];
    // make sure we use the correct protocol handler
    const client = url.startsWith("https") ? https : http;
    client
      .request(url, (conn) => {
        // aggregate the response stream into a single string.
        conn.on(`data`, (chunk) => data.push(chunk));
        conn.on(`end`, () => {
          // make sure to encode that string using utf8
          const asBytes = Buffer.concat(data);
          const asString = asBytes.toString(`utf8`);
          // and then trigger the resolution, with the
          // most frequently used fetch API "follow-up"
          // functions:
          resolve({
            arrayBuffer: async () => asBytes,
            json: async () => JSON.parse(asString),
            text: async () => asString,
          });
        });
        conn.on(`error`, (e) => reject(e));
      })
      .end();
  });
};

Which you can then use for whatever you need, using the normal fetch syntax you're used to from the browser:

const fs = require(`fs`);

// As per the note above: remove the following line for Node 18 and above
const fetch = require(`./that-code-shown-above.js`);

fetch(`https://placecats.com/00/300`)
  .then(res => res.arrayBuffer())
  .then(bytes => fs.writeFileSync(`kitten.jpg`, new Uint8Array(bytes)))
  .catch(e => console.error(e));

try {
  const response = await fetch(`https://jsonplaceholder.typicode.com/todos/1`);
  const data = await response.json();
  console.log(data);
} catch (e) {
  console.error(e);
}

// etc.

5 Comments

would be nice if you could show how to actually use the built in fetch module to download files, your link does not say (and trying to google answers only leads to outdated / 3rd party things)
Why? It's the same Fetch API as found documented on MDN (already linked in the post), and the second code block literally shows you how to download a file (a kitten picture, even) so I don't know what you think is still missing here =)
awww placekitten.com is down
Also writeFileSync can't write an ArrayBuffer directly, but you can add a new UInt8Array(arrayBuffer) around it - a typed array can be written as is.
good point, updated the answer.
7

Vince Yuan's code is great but it seems to be something wrong.

function download(url, dest, callback) {
    var file = fs.createWriteStream(dest);
    var request = http.get(url, function (response) {
        response.pipe(file);
        file.on('finish', function () {
            file.close(callback); // close() is async, call callback after close completes.
        });
        file.on('error', function (err) {
            fs.unlink(dest); // Delete the file async. (But we don't check the result)
            if (callback)
                callback(err.message);
        });
    });
}

1 Comment

can we specify the destination folder?
7
const download = (url, path) => new Promise((resolve, reject) => {
http.get(url, response => {
    const statusCode = response.statusCode;

    if (statusCode !== 200) {
        return reject('Download error!');
    }

    const writeStream = fs.createWriteStream(path);
    response.pipe(writeStream);

    writeStream.on('error', () => reject('Error writing to file!'));
    writeStream.on('finish', () => writeStream.close(resolve));
});}).catch(err => console.error(err));

Comments

7

I prefer request() because you can use both http and https with it.

request('http://i3.ytimg.com/vi/J---aiyznGQ/mqdefault.jpg')
  .pipe(fs.createWriteStream('cat.jpg'))

1 Comment

Looks like Request has been deprecated github.com/request/request/issues/3142 "As of Feb 11th 2020, request is fully deprecated. No new changes are expected to land. In fact, none have landed for some time."
7

✅So if you use pipeline, it would close all other streams and make sure that there are no memory leaks.

Working example:

const http = require('http');
const { pipeline } = require('stream');
const fs = require('fs');

const file = fs.createWriteStream('./file.jpg');

http.get('http://via.placeholder.com/150/92c952', response => {
  pipeline(
    response,
    file,
    err => {
      if (err)
        console.error('Pipeline failed.', err);
      else
        console.log('Pipeline succeeded.');
    }
  );
});

From my answer to "What's the difference between .pipe and .pipeline on streams".

Comments

7

Using the http2 Module

I saw answers using the http, https, and request modules. I'd like to add one using yet another native NodeJS module that supports either the http or https protocol:

Solution

I've referenced the official NodeJS API, as well as some of the other answers on this question for something I'm doing. The following was the test I wrote to try it out, which worked as intended:

import * as fs from 'fs';
import * as _path from 'path';
import * as http2 from 'http2';

/* ... */

async function download( host, query, destination )
{
    return new Promise
    (
        ( resolve, reject ) =>
        {
            // Connect to client:
            const client = http2.connect( host );
            client.on( 'error', error => reject( error ) );

            // Prepare a write stream:
            const fullPath = _path.join( fs.realPathSync( '.' ), destination );
            const file = fs.createWriteStream( fullPath, { flags: "wx" } );
            file.on( 'error', error => reject( error ) );

            // Create a request:
            const request = client.request( { [':path']: query } );

            // On initial response handle non-success (!== 200) status error:
            request.on
            (
                'response',
                ( headers/*, flags*/ ) =>
                {
                    if( headers[':status'] !== 200 )
                    {
                        file.close();
                        fs.unlink( fullPath, () => {} );
                        reject( new Error( `Server responded with ${headers[':status']}` ) );
                    }
                }
            );

            // Set encoding for the payload:
            request.setEncoding( 'utf8' );

            // Write the payload to file:
            request.on( 'data', chunk => file.write( chunk ) );

            // Handle ending the request
            request.on
            (
                'end',
                () =>
                {
                    file.close();
                    client.close();
                    resolve( { result: true } );
                }
            );

            /* 
                You can use request.setTimeout( 12000, () => {} ) for aborting
                after period of inactivity
            */

            // Fire off [flush] the request:
            request.end();
        }
    );
}

Then, for example:

/* ... */

let downloaded = await download( 'https://gitlab.com', '/api/v4/...', 'tmp/tmpFile' );

if( downloaded.result )
{
    // Success!
}

// ...

External References

EDIT Information

  • The solution was written for typescript, the function a class method - but with out noting this the solution would not have worked for the presumed javascript user with out proper use of the function declaration, which our contributor has so promptly added. Thanks!

2 Comments

Is this backward compatible? Works everywhere or only for http2?
@Neil I'm assuming you are asking if http2 is backward compatible with http1.2 - and the answer is no. Because http2 improves framing; adding binary compression, the ability to push from server to client, and simultaneous connections - it is absolutely required that both server and client know the implementation (this allows abstraction of implementation from the application too). Good news is that all major browsers have supported http2 since about 2015 - and Node as a client does too. Node, Nginx, and Apache offer it server side - so most use cases are covered. Its a vast improvement.
6

download.js (i.e. /project/utils/download.js)

const fs = require('fs');
const request = require('request');

const download = (uri, filename, callback) => {
    request.head(uri, (err, res, body) => {
        console.log('content-type:', res.headers['content-type']);
        console.log('content-length:', res.headers['content-length']);

        request(uri).pipe(fs.createWriteStream(filename)).on('close', callback);
    });
};

module.exports = { download };


app.js

... 
// part of imports
const { download } = require('./utils/download');

...
// add this function wherever
download('https://imageurl.com', 'imagename.jpg', () => {
  console.log('done')
});

Comments

6

2024 answer using async...await

Just use native fetch like:

const response = await fetch(url, {
   method: 'POST',
   body,
})
const buffer = await response.arrayBuffer()
fs.writeFileSync(`file${i}.txt`, Buffer.from(buffer))

My route sent a Response like:

return new Response(data, {
   headers: { 'content-type': 'application/text' },
})

so I used Buffer.from which you might or might not need depending on whether you want to download an image or text file.

Comments

5

Download using promise, which resolve a readable stream. put extra logic to handle the redirect.

var http = require('http');
var promise = require('bluebird');
var url = require('url');
var fs = require('fs');
var assert = require('assert');

function download(option) {
    assert(option);
    if (typeof option == 'string') {
        option = url.parse(option);
    }

    return new promise(function(resolve, reject) {
        var req = http.request(option, function(res) {
            if (res.statusCode == 200) {
                resolve(res);
            } else {
                if (res.statusCode === 301 && res.headers.location) {
                    resolve(download(res.headers.location));
                } else {
                    reject(res.statusCode);
                }
            }
        })
        .on('error', function(e) {
            reject(e);
        })
        .end();
    });
}

download('http://localhost:8080/redirect')
.then(function(stream) {
    try {

        var writeStream = fs.createWriteStream('holyhigh.jpg');
        stream.pipe(writeStream);

    } catch(e) {
        console.error(e);
    }
});

2 Comments

302 is also HTTP status code for URL redirect, so you should use this [301,302].indexOf(res.statusCode) !== -1 in the if statement
The questions was specific to not include third party modes :)
4

If you are using express use res.download() method. otherwise fs module use.

app.get('/read-android', function(req, res) {
   var file = "/home/sony/Documents/docs/Android.apk";
    res.download(file) 
}); 

(or)

   function readApp(req,res) {
      var file = req.fileName,
          filePath = "/home/sony/Documents/docs/";
      fs.exists(filePath, function(exists){
          if (exists) {     
            res.writeHead(200, {
              "Content-Type": "application/octet-stream",
              "Content-Disposition" : "attachment; filename=" + file});
            fs.createReadStream(filePath + file).pipe(res);
          } else {
            res.writeHead(400, {"Content-Type": "text/plain"});
            res.end("ERROR File does NOT Exists.ipa");
          }
        });  
    }

Comments

3

You can use https://github.com/douzi8/ajax-request#download

request.download('http://res.m.ctrip.com/html5/Content/images/57.png', 
  function(err, res, body) {}
);

2 Comments

It is returning garbage character if file name is other than ascii like if filename is in japanese.
Do you think ajax-request is not a third party library?
1

Writing my own solution since the existing didn't fit my requirements.

What this covers:

  • HTTPS download (switch package to http for HTTP downloads)
  • Promise based function
  • Handle forwarded path (status 302)
  • Browser header - required on a few CDNs
  • Filename from URL (as well as hardcoded)
  • Error handling

It's typed, it's safer. Feel free to drop the types if you're working with plain JS (no Flow, no TS) or convert to a .d.ts file

index.js

import httpsDownload from httpsDownload;
httpsDownload('https://example.com/file.zip', './');

httpsDownload.[js|ts]

import https from "https";
import fs from "fs";
import path from "path";

function download(
  url: string,
  folder?: string,
  filename?: string
): Promise<void> {
  return new Promise((resolve, reject) => {
    const req = https
      .request(url, { headers: { "User-Agent": "javascript" } }, (response) => {
        if (response.statusCode === 302 && response.headers.location != null) {
          download(
            buildNextUrl(url, response.headers.location),
            folder,
            filename
          )
            .then(resolve)
            .catch(reject);
          return;
        }

        const file = fs.createWriteStream(
          buildDestinationPath(url, folder, filename)
        );
        response.pipe(file);
        file.on("finish", () => {
          file.close();
          resolve();
        });
      })
      .on("error", reject);
    req.end();
  });
}

function buildNextUrl(current: string, next: string) {
  const isNextUrlAbsolute = RegExp("^(?:[a-z]+:)?//").test(next);
  if (isNextUrlAbsolute) {
    return next;
  } else {
    const currentURL = new URL(current);
    const fullHost = `${currentURL.protocol}//${currentURL.hostname}${
      currentURL.port ? ":" + currentURL.port : ""
    }`;
    return `${fullHost}${next}`;
  }
}

function buildDestinationPath(url: string, folder?: string, filename?: string) {
  return path.join(folder ?? "./", filename ?? generateFilenameFromPath(url));
}

function generateFilenameFromPath(url: string): string {
  const urlParts = url.split("/");
  return urlParts[urlParts.length - 1] ?? "";
}

export default download;

Comments

0

Path : img type : jpg random uniqid

    function resim(url) {

    var http = require("http");
    var fs = require("fs");
    var sayi = Math.floor(Math.random()*10000000000);
    var uzanti = ".jpg";
    var file = fs.createWriteStream("img/"+sayi+uzanti);
    var request = http.get(url, function(response) {
  response.pipe(file);
});

        return sayi+uzanti;
}

Comments

0

Without library it could be buggy just to point out. Here are a few:

Here my suggestion:

  • Call system tool like wget or curl
  • use some tool like node-wget-promise which also very simple to use. var wget = require('node-wget-promise'); wget('http://nodejs.org/images/logo.svg');

Comments

0
function download(url, dest, cb) {

  var request = http.get(url, function (response) {

    const settings = {
      flags: 'w',
      encoding: 'utf8',
      fd: null,
      mode: 0o666,
      autoClose: true
    };

    // response.pipe(fs.createWriteStream(dest, settings));
    var file = fs.createWriteStream(dest, settings);
    response.pipe(file);

    file.on('finish', function () {
      let okMsg = {
        text: `File downloaded successfully`
      }
      cb(okMsg);
      file.end(); 
    });
  }).on('error', function (err) { // Handle errors
    fs.unlink(dest); // Delete the file async. (But we don't check the result)
    let errorMsg = {
      text: `Error in file downloadin: ${err.message}`
    }
    if (cb) cb(errorMsg);
  });
};

Comments

0
var fs = require('fs'),
    request = require('request');

var download = function(uri, filename, callback){
    request.head(uri, function(err, res, body){
    console.log('content-type:', res.headers['content-type']);
    console.log('content-length:', res.headers['content-length']);
    request(uri).pipe(fs.createWriteStream(filename)).on('close', callback);

    }); 
};   

download('https://www.cryptocompare.com/media/19684/doge.png', 'icons/taskks12.png', function(){
    console.log('done');
});

Comments

0

Here's yet another way to handle it without 3rd party dependency and also searching for redirects:

        var download = function(url, dest, cb) {
            var file = fs.createWriteStream(dest);
            https.get(url, function(response) {
                if ([301,302].indexOf(response.statusCode) !== -1) {
                    body = [];
                    download(response.headers.location, dest, cb);
                  }
              response.pipe(file);
              file.on('finish', function() {
                file.close(cb);  // close() is async, call cb after close completes.
              });
            });
          }

Comments

0

Here is the easiest way to do this:

async function archive_fromURL(url: string, destinationPath: string) {

    const response = await axios.get(url, { responseType: 'arraybuffer' });
    const fileData = Buffer.from(response.data, 'binary')
    return fs.writeFile(destinationPath, fileData, (err) => {
        if (err) { console.log("ErrorMessage:", err.message) }
    })

}

Comments

0

Slightly modifying deadcoder0904's response for real beginners (i.e. me) that aren't even using [options] in Node's new fetch implementation:

const fs = require('node:fs');

let url = 'https://example.com/';
let fn = 'example.html';

let promise = fetch(url);
promise.then(response => response.blob())
   .then(blob => blob.arrayBuffer())
   .then(Buffer.from)
   .then((buf) => {
      fs.writeFileSync(fn, buf);
      console.log('file written');
      return;
   });

The above code works for images, text, and entire webpages, just by changing fn and url - from what I have tested. More explanation of each step would be appreciated.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.