0

I seem to be having an issue when trying to write to a file in a loop, the loop is iterating even though the first file has not been created (I think i am either not understanding promises or the asynchronous nature of the script)

So on the command line i will run node write_file_script.js premier_league

// teams.js
module.exports = {
premier_league: [
  { team_name: "Team 1", friendly_name: "name 1"},
  { team_name: "Team 2", friendly_name: "name 2"}
 ]
}

My Script

const args = process.argv.slice(2);
const TEAM = require('./teams');

const Excel = require('exceljs');
const workbook = new Excel.Workbook();

for (var team = 0; team < TEAM[args].length; team++) {
  console.log("Starting Excel File generation for " + TEAM[args][team]['team_name']);
  var fhcw = require('../data_files/home/fhcw/' + TEAM[args][team]['friendly_name'] + '_home' + '.json');
  fhcw = fhcw.map(Number);

  workbook.xlsx.readFile('./excel_files/blank.xlsx')
  .then(function() {
    var worksheet = workbook.getWorksheet(1);
    // Write FHCW
    for (i=0; i < fhcw.length; i++) {
      col = i+6;
      worksheet.getCell('E'+ col).value = fhcw[i];
    }

    console.log(TEAM[args][team])
    workbook.xlsx.writeFile('./excel_files/' + TEAM[args] + '/' + TEAM[args][team]['friendly_name'] + '.xlsx');

  });

}

The output i get when running this is

Starting Excel File generation for Team 1
Starting Excel File generation for Team 2
undefined
(node:75393) UnhandledPromiseRejectionWarning: Unhandled promise rejection 
(rejection id: 1): TypeError: Cannot read property 'friendly_name' of undefined

So it seems like the file is not being written but the loop continues, how can i ensure the file is written before moving onto the next loop?

Thanks

2
  • Have a look at the documentation for writing a file: github.com/guyonroche/exceljs#writing-csv You can see that there is a .then function which tells you when the writing of the file is done Commented Feb 7, 2018 at 10:32
  • Thanks, but seem to get same problem Commented Feb 7, 2018 at 10:42

2 Answers 2

1

If the function is returning a Promise, and you want to do them serially (one at a time) instead of in parallel (start all at the same time), you'll need to wait for each before you start the next using then().

Also, note that your TEAM is just an export of an array (at least as presented), so you can't give it args, which is where the other error comes from.

When you have a list of things to do, the best way to do them is to have a queue which you run until you run out of files. In this case, it looks like your TEAM array is your queue, but since this is an export, I'd recommend not necessarily changing this, but instead copy it to another array which you can alter:

const args = process.argv.slice(2);
const TEAM = require('./teams');

const Excel = require('exceljs');
const workbook = new Excel.Workbook();

const writeNextFile = (queue) => {
  // If we have nothing left in the queue, we're done.
  if (!queue.length) {
    return Promise.resolve();
  }

  const team = queue.shift(); // get the first element, take it out of the array

  console.log("Starting Excel File generation for " + team.team_name);
  var fhcw = require('../data_files/home/fhcw/' + team.friendly_name + '_home' + '.json');
  fhcw = fhcw.map(Number);

  // return this promise chain
  return workbook.xlsx.readFile('./excel_files/blank.xlsx')
  .then(function() {
    var worksheet = workbook.getWorksheet(1);
    // Write FHCW
    for (i=0; i < fhcw.length; i++) {
      col = i+6;
      worksheet.getCell('E'+ col).value = fhcw[i];
    }

    console.log(team);
    // not sure what you thought this would TEAM[args] would have translated to, but it wouldn't have been a string, so I put ??? for now
    // also, making the assumption that writeFile() returns a Promise.
    return workbook.xlsx.writeFile('./excel_files/' + team.??? + '/' + team.friendly_name + '.xlsx');
  }).then(() => writeNextFile(queue));
}

writeNextFile(TEAM.slice(0)) // make a copy of TEAM so we can alter it
  .then(() => console.log('Done'))
  .catch(err => console.error('There was an error', err)); 

Basically, our function takes an array and will write the first team, then call itself recursively to call the next. Eventually it'll all resolve and you're end up with is a Promise that resolves at the end.

When it comes to Promises, you basically always need to chain them together. You won't be able to use a for loop for them, or any other standard loop for that matter.

If you wanted to write them all at once, it is a little cleaner, because you can just do a non-recursive map for each and use Promise.all to know when they are done.

const writeNextFile = (team) => {
  console.log("Starting Excel File generation for " + team.team_name);
  var fhcw = require('../data_files/home/fhcw/' + team.friendly_name + '_home' + '.json');
  fhcw = fhcw.map(Number);

  // return this promise chain
  return workbook.xlsx.readFile('./excel_files/blank.xlsx')
  .then(function() {
    var worksheet = workbook.getWorksheet(1);
    // Write FHCW
    for (i=0; i < fhcw.length; i++) {
      col = i+6;
      worksheet.getCell('E'+ col).value = fhcw[i];
    }

    console.log(team);
    // not sure what you thought this would TEAM[args] would have translated to, but it wouldn't have been a string, so I put ??? for now
    // also, making the assumption that writeFile() returns a Promise.
    return workbook.xlsx.writeFile('./excel_files/' + team.??? + '/' + team.friendly_name + '.xlsx');
  });
}

Promise.all(TEAM.map(writeTeamFile))
  .then(() => console.log('Done')
  .catch(err => console.error('Error'));

Promises are for async, and generally when you have a group of things, you want to do them in parallel because it's faster. That's why the parallel version is a lot cleaner; we don't have to call things recursively.

Sign up to request clarification or add additional context in comments.

4 Comments

I have a question around your statement of Also, note that your TEAM is just an export of an array (at least as presented), so you can't give it args, which is where the other error comes from. if i run node script.js premier_league then my argument is premier_league which gives me access to that object within TEAM, for some reason i thought that it would translate to the string premier_league but that is obviously incorrect
Oh, sorry, I misunderstood what you were going for there. Yes, if args is 'premier_league, then you are correct that you should use args` there... sorta. Instead of process.argv.slice(2) just do process.argv[2] or else you'll have an array which wouldn't make sense. However, the bit where I did teams.??? still doesn't make sense. In that case, you'd probably just want args instead of teams[args].
well i have just added league: key now to the JSON object so can just reference team.league.. so your answer works (serial version) with a small amendment const writeNextFile = function(queue), was getting an error without.
Haha, it's no problem, really appreciate the time you took in answering in such a detailed way :-), just wish i could understand these things more
1

Since you are not writing to the same file, you can loop through the files and perform read/write operations.

You can bind the index value in the loop, and move ahead.

Example code.

var filesWrittenCount = 0;

for(var team = 0; team < TEAM[args].length; team++){
    (function(t){
        // call your async function here. 
        workbook.xlsx.readFile('./excel_files/blank.xlsx').then(function() {
            var worksheet = workbook.getWorksheet(1);
            // Write FHCW
            for (i=0; i < fhcw.length; i++) {
                col = i+6;
                worksheet.getCell('E'+ col).value = fhcw[i];
            }

            console.log(TEAM[args][t])
            workbook.xlsx.writeFile('./excel_files/' + TEAM[args] + '/' + TEAM[args][t]['friendly_name'] + '.xlsx');

            // update number of files written.
            filesWrittenCount++;

            if (totalFiles == filesWrittenCount) {
                // final callback function - This states that all files are updated
                done();
            }

        });
    }(team));
}

function done() {
    console.log('All data has been loaded');
}

Hope this helps.

4 Comments

writeFile() also returns a Promise, so it'd have to be inside of that. It also looks like they want write one file a time. If that's the case, this won't do that. It fires them all off in parallel, which there are cleaner approaches for.
If they are writing different files, I don't see a reason for serializing them. They might have a use-case for that.
Nor do I, but their question was about having one complete before the other. Even if you are going to parallelize with Promises, you should make use of Promise.all instead of counters and callbacks.
Thank you both for answering, im going to go through them and try to get some understanding. Im not sure it makes a difference at present if i do them parallel or serially as i don't have many files to create, however the more this increases in parallel will obviously have its benefits

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.