0

I'm using the fast-csv node library to parse a csv-file that looks like this

var stream = fs.createReadStream('./tmp_dir/'+files.fileUploaded.name);

var csvStream = csv()
    .on("data", function(data){
         console.log(data);
    })
    .on("end", function(){
         console.log("done");
    });

stream.pipe(csvStream);

I wonder if it is possible to do the following things

  • To see how many rows in the csv file that could not be parsed?
  • Record start and end time for parsing of the file?
Improve this question

2 Answers 2

Reset to default
1

You put the link to the library github but you didn't read the doc!

It's all in here !

var stream = fs.createReadStream('./tmp_dir/'+files.fileUploaded.name);
var errorCount = 0;
var csvStream = csv.parse({strictColumnHandling: true, headers: true})
.on("data", function(data){
     console.log(data);
}).on("data-invalid", function(){
     console.log("error");
     errorCount += 1;
})    
.on("end", function(){
     console.log("done");
});

stream.pipe(csvStream);

For the timing you can just get the date when you read the first line (use a counter or a toggle isFirstLine that you set to false after read) and when you hit the finish event, and you'll have timing.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

The event is called data-invalid.
And headers has to be set to true too I think.
0

1

To see how many rows in the csv file that could not be parsed?

In the docs, it is mentioned that the event data-invalid is emitted when invalid row-data is encountered. I'm not sure but perhaps you can use that.

var csvStream = csv()
  .on("data-invalid", function(data){
     // handle the erroneous data
  })
  .on("data", function(data){
     console.log(data);
  })
  .on("end", function(){
     console.log("done");
  });

2

Record start and end time for parsing of the file?

You could try this:

var startTime = (new Date()).getTime();

var csvStream = csv()
  .on("end", function(){
     console.log("done");

     var endTime = (new Date()).getTime();

     console.log("Time taken : ", (endTime - startTime));

  });
Improve this answer

1 Comment

the data-invalid event will not be triggered as mentioned in the docs, because you need to specify a strict handling in the options, see my answer below.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.