Node js : change csv file encoding programatically and parse to json

Question

I have a csv file with utf-16 encoding, and need to change its enconding to utf8 and convert it to JSON. I'm using csvtojson and iconv-lite modules. Here is my code:

var data = fs.createReadStream("myfile.csv");
data.pipe(iconv.decodeStream('utf16'))
  .pipe(iconv.encodeStream('utf8'))
  .pipe(fs.createWriteStream("encoded.csv"));
var Converter = require("csvtojson").Converter;
var csvStr = fs.readFileSync("encoded.csv").toString();
var converter = new Converter({});
converter.fromString(csvStr, function(err, jsonObj) {
    if (err) {
        handleError(err)
    }
    console.log(jsonObj)
});

The problem is that iconv converts the csv file with the right encoding, but when I read this file and call toString() method, it returns an empty string. How can I fix this?

dm03514 · Accepted Answer · 2015-09-03 13:02:02Z

5

I think asynchronous pipe might have something to do with it. You can ensure you only open encoded.csv after it is completed by putting logic in the end event.

var data = fs.createReadStream("myfile.csv");
data.pipe(iconv.decodeStream('utf16'))
  .pipe(iconv.encodeStream('utf8'))
  .pipe(fs.createWriteStream("encoded.csv"));

data.on('end', function() {

  var Converter = require("csvtojson").Converter;
  var csvStr = fs.readFileSync("encoded.csv").toString();
  var converter = new Converter({});
  converter.fromString(csvStr, function(err, jsonObj) {
    if (err) {
        handleError(err)
    }
    console.log(jsonObj)
  });
}

https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options

answered Sep 3, 2015 at 13:02

dm03514

56.1k18 gold badges117 silver badges147 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Node js : change csv file encoding programatically and parse to json

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related