-1

I am parsing an excel file, that has 7 columns and over 300k rows.

I need to create objects (Nodejs side) and make a bulk insert in the database.

I am wondering what would be the best way to create large amounts of objects for this purpose.

I have come up with three approaches, and not sure which one would be the nest in terms of speed and memory:

First:

var bulkObjects = [];
worksheet.eachRow({includeEmpty: true}, function (row, rowNumber) {
    var currentObject = createRowObject(row.values);    
    bulkObjects.push(currentObject);
});

function createRowObject(row) {    

    return {
        Row1: row[1],
        Row2: row[2],
        Row3: row[3],
        Row4: row[4],
        Row5: row[5],
        Row6: row[6]
     }
}

Second:

var bulkObjects = [];
worksheet.eachRow({includeEmpty: true}, function (row, rowNumber) {
    var currentObject = **new** createRowObject(row.values);    
    bulkObjects.push(currentObject);
});

function createRowObject(row) { 

    this.Row1: row[1],
    this.Row2: row[2],
    this.Row3: row[3],
    this.Row4: row[4],
    this.Row5: row[5],
    this.Row6: row[6]    
}

Third:

var bulkObjects = [];
worksheet.eachRow({includeEmpty: true}, function (row, rowNumber) {
    var currentObject = createRowObject(row.values);
    bulkObjects.push(currentObject);
});

function createRowObject(row) {

    var o = new Object();
    o["Row1"] = row[1];
    o["Row2"] = row[2];
    o["Row3"] = row[3];
    o["Row4"] = row[4];
    o["Row5"] = row[5];
    o["Row6"] = row[6];

    return o;
}

Which one would be the best to create large amount of objects. Is there a another approach?

4
  • why you don't set up a test and check with Profiling JavaScript Performance? Commented Apr 15, 2016 at 12:35
  • Possible duplicate of Which way is best for creating an object in javascript? is "var" necessary before variable of object? Commented Apr 15, 2016 at 12:36
  • 1
    If memory is an issue, the best way would be to do multiple inserts of batches of bulkObjects instead of doing it all in one go, so all the objects that have been inserted already, can be garbage collected. Concerning the object creation, since there are no methods on the object all 3 of your approaches should be around the same speed, so go for the simpler one, the first. You can even make it simpler by removing the function call and just pushing an object directly. Commented Apr 15, 2016 at 12:45
  • Is there an example how to do multiple bulk inserts? Commented Apr 15, 2016 at 12:57

1 Answer 1

0

I suggest if you are using node js on server side then you should try Loadash library for faster and error free object transformations.

Try this

var objectMappingHeaders = ['Row1','Row2','Row3','Row4','Row5','Row6','Row7'];
var data = [[1,2,3,4,5,6,7],['a','b','c','d','e','f','g']];

var output = _.reduce(data, function(result, val) {
  result.push(_.zipObject(objectMappingHeaders, val));
  return result;
}, [])
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.