How should I return data from fs.createReadStream() using csv parser

Question

I have this function

function getData(file, type) {
    let data = [];
    fs.createReadStream(file)
        .pipe(csv({headers: false, separator: ';',}))
        .on('data', (row) => {
            let item = {
                date: row[0],
                value: row[1]
            };
            let item2 = {
                date: moment(row[0], "DD-MM-YYYY HH:mm").add(30, "minutes").format("DD/MM/YYYY HH:mm"),
                value: row[2]
            };
            data.push(item);
            data.push(item2);
        })
        .on('end', () => {
           return data;
        });
}

and I would like to return the data parsed in the csv, but when I return data in the .on method, data is empty.

How should I do it ?

You should listen to .on('error', (err) => { see if you have no issue. Also put the .on('data', (row) => { function into a try/catch — Orelsanpls
– Orelsanpls, Commented Jan 13, 2020 at 10:17
What I do right now, instead of returning data, is using a global var, so I know there is no error, right now. So, it doesn't fix the issue, but anyway, I will check it, it is an important variable, you are right ! — Juliatzin
– Juliatzin, Commented Jan 13, 2020 at 10:24

Terry Lennox · Accepted Answer · 2020-01-13 12:39:24Z

21

I would suggest using a promise-based approach, you can return a promise from the getData function, then resolve or reject as appropriate.

This keeps things a lot cleaner for the calling function.

For example:

function getData(file, type) {
    let data = [];
    return new Promise((resolve, reject) => {
        fs.createReadStream(file)
            .on('error', error => {
                reject(error);
            })
            .pipe(csv({headers: false, separator: ';',}))
            .on('data', (row) => {
                let item = {
                    date: row[0],
                    value: row[1]
                };
                let item2 = {
                    date: moment(row[0], "DD-MM-YYYY HH:mm").add(30, "minutes").format("DD/MM/YYYY HH:mm"),
                    value: row[2]
                };
                data.push(item);
                data.push(item2);
            })
            .on('end', () => {
                resolve(data);
            });
    });
}

async function testGetData() {
    try { 
        const data = await getData("test.csv", {});
        console.log("testGetData: parsed CSV data:", data);
    } catch (error) {
        console.error("testGetData: An error occurred: ", error.message);
    }
}

testGetData();

answered Jan 13, 2020 at 12:39

Terry Lennox

30.9k5 gold badges36 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

KD.S.T. Over a year ago

niiiiiice! way to go

Collectives™ on Stack Overflow

How should I return data from fs.createReadStream() using csv parser

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related