Importing multiple CSV files in Javascript

Question

I need to import 8-10 CSV files into some JS code. Each file has about 8 different columns. I would like to store this information in an object (possibly identifying the object based on one of the ID fields). The problem is, the easiest way I can think of looks like this:

var authorArr = [];
d3.csv("data/authors.csv", function(csv) {
    csv.forEach(function(d) {
    authorArr[d.record_id] = [];
    d.record_id = +d.record_id;
    d.name = d.name
    d.name_inverted = d.name_inverted;
    d.role = d.role;
    d.sequence = +d.sequence;
    d.is_person = d.is_person;

    authorArr[d.record_id] = d.name;
    authorArr[d.record_id] = d.role;
    etc...

Surely this cannot be the quickest way...Also, there are quite a bit of duplicated record_id values as well, so every time there is a repeat, I would lose all previous data with my horrible approach.

I'm not sure if that's enough detail. If not, I would be glad to add more.

Juve · Accepted Answer · 2012-10-24 21:37:05Z

1

You can use a temporary variable for the new/current record. Your example also looks like you do not want an array (with gaps) but a map. Here is how I would load the data:

var authorMap = {}

var loadCSV = function(file){
    d3.csv(file, function(error, data) {
        data.forEach(function(d) {
            var a = authorMap[d.record_id] = {}
            a.record_id = d.record_id
            a.name      = d.name
            a.role      = d.role
        });
    });
}

loadCSV("file1.csv")
loadCSV("file2.csv")

answered Oct 24, 2012 at 21:37

Juve

10.9k14 gold badges69 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

1080p Over a year ago

I am still quite the new programmer, so I don't know too much about maps. I will tinker with your approach and see what I think. Is there any way to automate the identifier? (ex: instead of asking for d.name and calling it d.name, can I just name each column by it's column name) Thanks a lot for the help!

Juve Over a year ago

I guess you first need to learn more about Javascript. About arrays vs objects and such stuff. :-)

Collectives™ on Stack Overflow

Importing multiple CSV files in Javascript

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related