Replace all text within two strings on different lines using Perl

Question

I am a web developer. The agency I work for uses Dreamweaver for its templating/library item features. The library items really come in handy for updating a nav bar or some piece of content thats the same on every page. We work with static HTML most of the time. So to change a menu item, we use a dreamweaver library item, we update that item once, we press update, and it changes it across every html page in the project.

But I would like to make a perl script I can run from the command line instead of opening up a GUI, it would simply be faster.

So for example, lets say I have a menu coded like this:

<!--MENUITEMS-->
<li><a href="products.html">Products</a></li>
<li><a href="about_us.html">About Us</a></li>
<li><a href="commercial.html">Commercial</a></li>
<li><a href="contact.html">Contact</a></li>
<!--MENUEND-->

I will store the li items code in their own file: nav.lbi:

<li><a href="products.html">Products</a></li>
<li><a href="about_us.html">About Us</a></li>
<li><a href="commercial.html">Commercial</a></li>
<li><a href="contact.html">Contact</a></li>

The perl script needs to replace in each file it scans all the text between  and  with the contents of nav.lbi.

I was going to first try this in SED, but SED is tailored for line by line stuff. I have had success using SED to insert an entire text file somewhere in another file, but this is a bit different. With Perl I know I should be able to replace all the text between every occurance of  and  with the contents of nav.lbi, even though it spans multiple lines.

If I need to add the  and  tags to the actually nav.lbi file, since its doing a search and replace, if that would make it easier, that is fine too. This is just so I can update the navigation bar across multiple html files without needing to touch Dreamweaver.

One last thing to note is there is multiple occurances of  and the closing  because the navigation in the header is often identical to the navigation in the footer, so I need to update the file recursively.

I am uncomfortable with aiding what sounds like a simple prejudice against perfectly satisfactory Windows tools. Yes, Perl will do what you ask, and it may be exactly what Dreamweaver does, but I would hope to encourage you towards something better rather than replicating the same poor design on anything non-Windows. If I have misunderstood you then please explain — Borodin
– Borodin, Commented Jul 22, 2015 at 18:07
It sounds like what you're doing is writing your own templating system, but Perl already has a number of templating systems for you to choose from. I suggest you start with Template Toolkit. template-toolkit.org — Andy Lester
– Andy Lester, Commented Jul 22, 2015 at 18:36
@Borodin - My thing against Adobe products is they are very processor intensive. And running Linux I have to run Dreamweaver in Virtual Machine, which is more processor intensive. And since I like to program, why wouldn't I take the time to learn how to do it in code instead of relying on third party software which costs money??? The command line is also faster. Using ImageMagick instead of Adobe saves me lots of time. Not having to open up Dreamweaver (after booting Windows VM) just to update the nav when I can run a script is more ideal to a Linux programmer. — David Provost
– David Provost, Commented Jul 22, 2015 at 18:39

David Provost · Accepted Answer · 2015-07-26 03:28:28Z

1

Just add a s at the end of the regular expression to make dot match new lines as well. See man perlre:

# or use File::Slurp;
sub slurp {
    my $fn = shift;
    open(IN,$fn);
    return join('',<IN>);
}

my $_ = slurp("in.html");
my $new_menu_items = slurp("nav.lbi");

s/<!--MENUITEMS-->(.*)<!--MENUEND-->/$new_menu_items/s;
print;

To save the output to a new file rather than just printing it, you need to specifiy the output file, and print to it, so the above code would become:

sub slurp {
     my $fn = shift;
     open(IN,$fn);
     return join('',<IN>);
 }   

 my $_ = slurp("in.html");
 my $new_menu_items = slurp("nav.lbi");
 open my $output_file, '>', 'output.html';

 s/<!--MENUITEM-->(.*)<!--MENUEND-->/$new_menu_items/s;
 print $output_file $_;

For more information on slurping in Perl:
http://www.perl.com/pub/2003/11/21/slurp.html

edited Jul 26, 2015 at 3:28

David Provost

4374 silver badges11 bronze badges

answered Jul 22, 2015 at 18:57

Dov Grobgeld

5,0212 gold badges29 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

David Provost Over a year ago

How can I write back out the file rather than print?

David Provost Over a year ago

How do I run this on multiple files at once? i.e. - I want to script to update each .html page in directory with the nav.lbi.

Dov Grobgeld Over a year ago

If you are going to use a language, I suggest that you take the time to learn it. ;-) At least basic things like opening files for reading and writing. Besides I see that someone already elaborated on my answer. And I who was going to suggest a function called burp() doing the opposite of slurp(). :-)

OB7 · Accepted Answer · 2015-09-30 16:29:27Z

The following builds upon Dov Grobgelds answer, but has the additional functionality added asked about in the comments to his answer. Precisely it runs the multi line find and replace on all html files in the directory and overwrites the original file:

#!/usr/bin/perl

use strict;
use warnings;

sub slurp {
    my $fn = shift;
    open(IN,$fn);
    return join('',<IN>);
}

my @files = grep ( -f ,<*.html>);
for my $file (@files) {
    print "$file\n";
    my $_ = slurp("$file");
    my $new_menu_items = slurp("nav.lbi");
    open my $output_file, '>', "$file";
    s/<!--MENUITEMS-->(.*)<!--MENUEND-->/$new_menu_items/s;
    print $output_file $_;
}

Explanation:
The first question was how to print the output to a file, rather than just the terminal. (refer to Dov Grobgelds answer to compare the original code)
1.Specifiy what that output file will be:

open my $output_file, '>', 'output.html';

Then print its output to the file:
print; - from original answer
becomes:

print $output_file $_;

Now to make your script run on all the html files in the directory, we need to turn this script into something more dynamic. We need to store all .html files in vars in an array, and then iterate through the array, storing each .html to var, then running the slurp/regex code on each file.
First declare the variable that holds the html files and populate it with .html files in current directory:

my @files = grep ( -f ,<*.html>);

The rest of the magic happens by wrapping the original slurp and regex code in a for loop:

#for each file in files array, file is stored via my $file:
for my $file (@files) {
    #unecessary put prints out file filename loop is working on:
    print "$file\n";
    #default var becomes current file contents:
    my $_ = slurp("$file");
    #this part isn't dynamic, its same each loop, there might be a better
    #place to put this:
    my $new_menu_items = slurp("nav.lbi");
    #makes your output filename the same as the input:
    open my $output_file, '>', "$file";
    #runs the regex replace on the file contents of orig file
    s/<!--MENUITEMS-->(.*)<!--MENUEND-->/$new_menu_items/s;
    #saves the contents back out to the original file 
    #(overwriting it with original data loaded after running regex on it:)
    print $output_file $_;
}

Now lets say your closing tag is not unique, as in a Dreamweaver template, you can add ? to (.*) to match only to the first occurance, instead of the last:

s/<!--MENUITEMS-->(.*?)<!--LIBRARYITEMEND-->/$new_menu_items/s;

Collectives™ on Stack Overflow

Replace all text within two strings on different lines using Perl

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related