Trying to use node.js module, 'replace-in-file', to find and replace multiple strings in a file

Question

I am using the 'replace-in-file' module for my node.js project. I have written the node module and app.js file below to (1) take form data from a webpage, (2) create a new file from a template, (3) iterate over each key/value pair in the data object, (4) for each key value pair, search the new file for a string matching the key and replace with the key's corresponding value, and (5) return the new file's name to the client.

When the app runs, a new file is created, but the only search and replace reflected in the new file appears to be the last search and replace run by the .forEach() loop.

Why aren't all of my search and replaces showing up in the new document?

Here is the module I wrote called make-docs.js:

var fs = require('fs');
var replace = require('replace-in-file');
var path = require('path');

// function to build agreement file
exports.formBuild = function(data){

  var fileName = data.docType + Date.now() + '.html';
  var filePath = __dirname + '/documents/' + fileName;

  // make a new agreement
  fs.createReadStream(data.docType + '.html').pipe(fs.createWriteStream(filePath));

  Object.keys(data).forEach(function(key){

    var options = {
      //Single file
      files: filePath,
      //Replacement to make (string or regex)
      replace: key,
      with: data[key].toUpperCase(),
      //Specify if empty/invalid file paths are allowed, defaults to false.
      //If set to true these paths will fail silently and no error will be thrown.
      allowEmptyPaths: false,
    };
    replace(options)
    .then(changedFiles => {
      console.log('Modified files:', changedFiles.join(', '));
      console.log(options);
    })
    .catch(error => {
      console.error('Error occurred:', error);
    });
  })
}

var express = require("express");
var fs = require("fs");
var bodyParser = require("body-parser");
var makeDocs = require("./make-docs.js");
var path = require('path');
var app = express();

app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.use('/documents', express.static(path.join(__dirname, 'documents')));

app.post("/docs", function(req, res) {
  console.log(req.body);
  res.send(makeDocs.formBuild(req.body));
});

app.listen(8080,function(){
  console.log("Node server listening on port 8080");
});

Adam's diagnosis is correct. But I was expecting he would tell you how to do asynchronous tasks in series. This is something you will probably need to learn to do, even if you don't need it in this case. There are npm modules which allow you to combine asynchronous tasks in various ways (parallel, series, or somewhere in between). async and bluebird are two such modules. You've used the promises syntax, so you may be more comfortable with bluebird. — David Knipe
– David Knipe, Commented Jan 18, 2017 at 22:16
I agree with David, and I would highly recommend bluebird if you want to work with promises. I'll update my answer to include an example on how you would do it with the promise interface and wait for all promises to complete. — Adam Reis
– Adam Reis, Commented Jan 19, 2017 at 4:48
Cool, I'm learning a lot from posting this question! I'm reading about Bluebird now. I hope to work it into my application as I work to refine my code. Thanks for the input! — HockeyMonkey
– HockeyMonkey, Commented Jan 19, 2017 at 21:53

Adam Reis · Accepted Answer · 2017-01-19 04:55:55Z

This is because you are firing off several replacements synchronously, while they happen asynchronously in the background. So what happens is that every replace operation reads the file contents, which are unchanged at the start, and then replacements are made but only one replacement is ever saved to the file (the last one).

To circumvent this particular problem in your setup, is to use synchronous replacement via the replace.sync API:

try {
  let changedFiles = replace.sync(options);
  console.log('Modified files:', changedFiles.join(', '));
}
catch (error) {
  console.error('Error occurred:', error);
}

Note however that this will block the execution of your script (and slow down your request) while all replacements are made. For small files and few replacements this shouldn't be a problem, but for larger files and more replacements it could be. So it would be recommended to use background processes to do these replacements, but that's beyond the scope of this answer.

Your use case of making multiple different replacements was not yet supported by the replace-in-file package, but I've added this feature in as it seems useful.

So now you can make multiple replacements at once (synchronously or asynchronously):

const replaceInFile = require('replace-in-file');

const replace = [/replace/g, /also/g];
const with = ['something', 'else'];
const files = [...];

replaceInFile({files, replace, with})
  .then(changedFiles => { ... })
  .catch(error => { ... });

You can populate the replace and with arrays from your object with key/value pairs. If you want to replace multiple values with the same replacement, you can use a string for replace.

Hope this helps.

For completeness sake, here's how you would process the replacement asynchronously in series one by one:

const Promise = require('bluebird');
const replace = [/replace/g, /also/g];
const with = ['something', 'else'];
const file = '...';

Promise
    .map(replace, (replacement, i) => {
        return replaceInFile({
            files: file,
            replace: replacement,
            with: with[i],
        });
    })
    .then(() => { 
        //read file now which will contain all replaced contents
    });

Note that this is a simplified example, and assumes the replace and with arrays are of the same size. Have a look at the Bluebird library for more details on how to work with promises in series/parallel etc.

Collectives™ on Stack Overflow

Trying to use node.js module, 'replace-in-file', to find and replace multiple strings in a file

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related