Express.js - Execute function on every HTTP request

Question

I have a function unzip() on my local Node.js server that downloads a .json.gz file from an AWS CloudFront URL and unzips it to .json, when the server is run with node app.js.

The posts from that json file are loaded before responding to routes -

app.all('*', function(req, res, next){
    fs.readFile('s3posts.json', function(err, data){
        res.locals.posts = JSON.parse(data);
        next();
    });
});

The problem is that the file is downloaded and unzipped when the server starts running, and serves that same data to the user as long as it's running. The file is not re-downloaded unless the server is restarted. So, any changes to that data is not reflected until the restart.

I've tried calling the unzip() function inside of the above mentioned app.all() route before the fs.readFile() function that reads from the json file, but that throws an error when I request any page -

undefined:1
undefined
^

SyntaxError: Unexpected token u in JSON at position 0
    at JSON.parse (<anonymous>)
    at ReadFileContext.callback (/Users/Anish/Workspace/NodeJS/unzipper/app.js:48:27)
    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:359:13)

When I look in the directory, I see that it downloaded the file but failed to unzip it. Even the downloaded file is corrupt and can't be unzipped manually.

NOTE - The file downloads, unzips and gets served fine if it's placed outside of app.all().

How can I download and unzip the file on every request from the user?

eavichay · Accepted Answer · 2017-04-06 18:07:44Z

1

I recommend reading about middlewares in https://expressjs.com/en/guide/using-middleware.html

app.use(function(request)... should help you deal with all request regardless of the routing.

answered Apr 6, 2017 at 18:07

eavichay

5172 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Anish Sana Over a year ago

Based on your suggestion, I tried using - app.use(function(req) { unzip(); }); @eavichay The file is downloaded and unzipped successfully, but my pages aren't being served. The browser just keeps waiting. I tried using the next parameter and placed the next() function after unzip() but I'm getting the error - TypeError: next is not a function. Any idea why?

eavichay Over a year ago

you should call next() callback. The middlewares do exactly what you want, and pass on the request. The function get a request, response and next callback arguments, and you should use the next if you want the server to go on with the request, or use next(false) if you want to drop it.

Anish Sana · Accepted Answer · 2017-04-06 21:09:55Z

1

I was able to use the request module to do this -

app.all('*', function(req, res, next) {
    request({
        method: 'GET',
        uri: 'http://post.s3post.cf/s3posts.json.gz',
        gzip: true
    }, function(error, response, body) {
        res.locals.posts = JSON.parse(body);
        next();
    });
});

This also eliminated the need for me to have a custom unzip() function that downloaded and unzipped the file.

answered Apr 6, 2017 at 21:09

Anish Sana

5861 gold badge12 silver badges36 bronze badges

Comments

Sunil Prajapati · Accepted Answer · 2017-04-06 18:11:24Z

0

You should try this,

app.all('*', function(req, res, next){
    fs.readFile('http://post.s3post.cf/s3posts.json.gz', function(err, data){
        res.locals.posts = JSON.parse(data);
        next();
    });
});

answered Apr 6, 2017 at 18:11

Sunil Prajapati

3552 silver badges6 bronze badges

Collectives™ on Stack Overflow

Express.js - Execute function on every HTTP request

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related