storing PDF files in mongoose using Node.js and Angular 6

Question

I have to store large amount of data in mongoose collection and it is getting really big and hard to handle. I want to save PDF files in somewhere and link it in the mongoose collection, I am not really sure for the best solution. Since most links are about direct file upload where as I am using pdfmake library which then stores to collection.

Here is my current model

annual.js

customerReport: {
    createdBy: String,
    creationDate: String,
    hour: Number,
    waitingHour: Number,
    ccEmails: Array,
    sendToEmail: String,
    pdf: Schema.Types.Mixed,
    customerNotes: String
  },

this pdf has whole pdf content inside.

and in my route when I save and store the data finally

router.post('/stage/complete', (req, res, next) => {
  var completed = new AnnualStage({
    specs: req.body.specs,
    contactDetails: req.body.contactDetails,
    CalculationData: req.body.calculationData,
    customersData req.body.customersData
    customerReport: req.body.customerReport,
  });
  completed
    .save()
    .then(result => {
      res.status(200).json({
        result: result
      });
    })
    .catch(err => {
      res.status(500).json({
        message: err.message
      });
    });
});

All the front end is done by Angular 6 and generates all data and pdf and save it to mongo collection

you can see inside I have added GridFs inside the route but where it will store the files and how it will link back to DB?

Sergeon · Accepted Answer · 2019-05-31 08:11:04Z

Disclaimer: This should be a comment but is too long for a comment.

Your code has a really serious flag that it's worth to mention:

You should under no circumstances call fs.readFileSync (or any synchronous method that involes I/O operations) inside an http controller.

This will block all of your users requests until the file has been read.If several users are calling this endpoint at the same time, most of them will get their request delayed for a lot of time.

Instead, you should call fs.readFile. With async/await and util.promisify the code will look almost the same, without blocking the main thread:

const util = require('util');
const readFile = util.promisify(fs.readFile.bind.fs); // not really sure the bind is really needed tbh, maybe util.promisify(fs.readFile) works too
router.post('/stage/complete', async (req, res, next) => {
  var completed = new AnnualStage({
    specialRequest: req.body.specialRequest,
    contactDetails: req.body.contactDetails,
    requestData: req.body.requestData,
    customerRequests: req.body.customers,
    customerReport: req.body.customerReport,
  });
  completed.customerReport.pdf.data = await readFile(req.body.customerReport);
  completed.customerReport.pdf.contentType = 'application/pdf';
  completed
    .save()
    .then(result => {
      res.status(200).json({
        message: 'Handling POST request to /completed',
        result: result
      });
    })
    .catch(err => {
      res.status(500).json({
        message: err.message
      });
    });
});

This change will seriously improve your post handler.

Now, your question. I'm really not sure about what you're asking about. You say:

you can see inside I have added GridFs inside the route but where it will store the files and how it will link back to DB?

There is no reference to gridFs in your code snippets. Maybe you forgot to paste some code? It is not clear now AnnualStage.save() actually saves in mongo. Does it uses gridfs?

It is not clear if your code is not working, or if you're asking about your solution being good enough.

Hi. Great Explanation. I have removed some code from the above to clear the things out. My concern is when PDF is generated in the client side it stores directly inside the mongoose document. Instead I wanted to reduce the doc size and store the files in more elegant way and link it back to document. Most sites are about storing images through upload method.

Collectives™ on Stack Overflow

storing PDF files in mongoose using Node.js and Angular 6

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related