Import huge json file into mongodb with nodejs

Question

i want to do a node.js script that goes through my mongo.db and checks if the url doesn't match with the url in my other database. As of right now, i have json with over 750k scam websites and i need to import it somehow to the database. Does anyone know any quick way to do it? I tried:

  for (var i = 0; i < files.length; i++) {
    var file = files[i];
    var link = require("./files/" + file);
    var newLink = new rescamdb({
      link: link,
      createdAt: Date.now(),
    });
    newLink.save();
    console.log(`
        🔧 Added new link to database
                  `);
  }

But it doesn't work, won't even create the database and then it fails on validation error. The json file isn't mongodb ready, it looks like this:

[
    "scamsite1.com",
    "scamsite2.com",
    "anotherscamsite.com",
    "scamsite4.com",
...

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. — Community
– Community Bot, Commented Jun 20, 2022 at 15:39
You need to share your model, and JSON file else it's hard to tell what you are trying to achieve here. — devAgam
– devAgam, Commented Jun 21, 2022 at 15:06

Zach Jensz · Accepted Answer · 2022-06-22 06:54:21Z

1

Converted my JSON so it looked like:

[
{"link":"somescamwebsite.com"},
...
]

and then imported it to the mongo by mongoDB Compass

edited Jun 22, 2022 at 6:54

Zach Jensz

4,0986 gold badges19 silver badges32 bronze badges

answered Jun 21, 2022 at 16:24

Peter

255 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

server_unknown Over a year ago

Are you still interested in a node.js script that will populate a database from a json file, or are you satisfied with using mongoDB compass to do this?

Collectives™ on Stack Overflow

Import huge json file into mongodb with nodejs

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related