How to fetch data from an api and save to mongodb using mongoose

Question

This is how I have defined my schema

const apiSchema = new Schema({
    entries: [{
        API: {
            type: String
        },
        Description: {
            type: String
        },
        Link: {
            type: String
        },
        Category: {
            type: String
        }

    }]
});

module.exports = mongoose.model('api', apiSchema);

And here is my controller

const Data = require('./models/api');
p.get('/', (req, res, next) => {
    request('https://api.publicapis.org/entries', function (error, response, body) {
        var result = JSON.parse(body);
        console.log('body:', result); 
        

result = new Data({
            entries: {
                API: req.body.API,
                Desription: req.body.Desription,
                Link: req.body.Link,
                Category: req.body.Category   
            }
         })
        result.save()
            .then(result => {
                console.log('Entry saved');
            })
            .catch(err => {
                console.log(err);
            });
        
    });
    
});

When I run the server and open compass I find that only the object id in the entries array gets saved. I want all the fields in the entries array to be saved in the database.

but if I understood correctly, in the code you have given above, you don't save fetched data, is that true? You simply save the data of the request object — Abdurrahim Ahmadov
– Abdurrahim Ahmadov, Commented May 16, 2022 at 9:06

Abdurrahim Ahmadov · Accepted Answer · 2022-05-16 13:08:25Z

2

You can save the fetched data as follows:

first import the HTTPS module to send the HTTPS get request
create an array to keep buffer chunks
When all chunks are completely received, concat these chunks
save concatenated data on the DB

//test.js controller file
const Data = require('./../database/models/test');
const https = require('https');

module.exports = (req,res)=>{
    let data =[]; //array to keep data chunks
    
    https.get('https://api.publicapis.org/entries', function (response) { //send  request to api
        
  response.on('data', d => {
    data.push(d); //get data as chunk by chunk and push them to array
  }).on('error',e=>{
      console.log(e); //
  });

  response.on('end',()=>{
//when all data chunks are received, 
// concat all buffered chunks and 
//create js object from it
   let fetchedData= JSON.parse(Buffer.concat(data).toString()); 
   console.log(fetchedData);
   let result = new Data({
    entries:fetchedData.entries // fetched data has entries array , that is data that you want to save
});
result.save() //save data to db
    .then(result => {
        console.log('Entry saved');
    })
    .catch(err => {
        console.log(err);
    });

    res.send(result);
  })

      
}) };

edited May 16, 2022 at 13:08

answered May 16, 2022 at 12:56

Abdurrahim Ahmadov

5715 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Saliko Over a year ago

Is this the conventional way of doing it. Should I always use https module?

Abdurrahim Ahmadov Over a year ago

No, because you have used request function and this function is the core node function, I have used `https.get method. But you can use the 3rd party library for example Axios, fetch, and so on.

Collectives™ on Stack Overflow

How to fetch data from an api and save to mongodb using mongoose

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related