sed command doesn't work in shell script

Question

I have a shell script that looks like the following:

#!/bin/bash
for file in $1/*.html; do
   echo "Working on $file ..."

   # delete headers in html files.
   sed -n '1,9d' $file

   # delete the last 4 lines of the file.
   sed -n -e :a -e '1,4!{P;N;D;};N;ba'

   # ant runs regex task on all apt files
   ant -Dtarget.file=$file

   # all .html are changed to .md
   mv $file `echo $file | sed 's/\(.*\.\)html/\1md/'` ;

done

but the script hangs on the first sed command and I have to force exit. I'm wondering if there's anything wrong about the way I've set this up?

If I remove the first two sed commands, I can run the other parts of the script, including that final one with a sed command in it.

The mv makes me cringe due to the useless and expensive fork to yet another sed. If you just want to replace an extension in a filename stored in a variable, use mv $file ${file%.html}.md. — Jens
– Jens, Commented May 12, 2012 at 16:55

paxdiablo · Accepted Answer · 2011-07-20 01:36:03Z

4

I think you'll find it's hanging on the second sed, where you don't provide a file name to work on. In other words, it will take its standard input from the terminal.

You won't get output from the first sed since -n prevents the default printing of the pattern space. If you're trying to change the actual file rather than output the changes to standard output, you need the -i inplace editor.

If you can't use the inplace option, you'll have to do something like:

sed -n '1,9d' ${file} >${file}.tmp0
sed -n -e :a -e '1,4!{P;N;D;};N;ba' ${file}.tmp0 >${file}
rm -f ${file}.tmp0

This uses a temporary file for making the changes except for the last change, which overwrites the original file. Keep in mind that this changes the original - if that's not what you want, you can change the second sed to output to ${file}.tmp2 but you'll have to adjust your subsequent commands to take that into account.

If you really want to delete the first nine and last four lines of a file, you can also use something like:

head --lines=-4 oldfile.txt | tail --lines=+10 >newfile.txt

provided you have a [rd]ecent implementation of head/tail. The head prints all but the last four lines and the tail starts printing at line ten.

edited Jul 20, 2011 at 1:36

answered Jul 19, 2011 at 3:34

paxdiablo

888k243 gold badges1.6k silver badges2k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user766353 Over a year ago

You are right, it didn't hang when I added the filename. However, still no lines were deleted from the head or tail of the files. I unfortunately do not have a decent head ... I mean the shell command :) ... but I managed to use tail for getting rid of the headers and ant-based regex for snipping the lines at the end.

paxdiablo Over a year ago

@user766353, no lines were deleted since you're not replacing the file with the output of sed. sed normally writes the changes to standard output (rather than changing the original file). What you'd normally see on the output is those changes but, since you've used -n to suppress the default pattern space output, they won't appear. See my update for a solution (making sure the file itself is updated).

Foo Bah · Accepted Answer · 2011-07-19 03:32:28Z

0

What command are you running? If nothing matches the file glob, then it will physically insert the unexpanded string there.

For example, try this:

$ for i in foo/*.bar; do echo $i; done
foo/*.bar

So your sed script is bombing out because the file name it is receiving doesn't exist, which means it will act as if no file argument was specified (ie stdin).

answered Jul 19, 2011 at 3:32

Foo Bah

26.4k5 gold badges58 silver badges82 bronze badges

1 Comment

paxdiablo Over a year ago

If the file doesn't exist, I'm pretty certain sed doesn't act as if you specified no file, instead it complains bitterly :-)

James Polley · Accepted Answer · 2011-07-19 03:34:46Z

0

Change the first line to:

#!/bin/bash -x

This will show you each command as bash executes it, after all expansions have taken place.

My guess is that $1/*.html isn't matching any files; hence the sed line evaluates to just sed -n '1,9d' and so sed is trying to read from stdin

answered Jul 19, 2011 at 3:34

James Polley

8,2413 gold badges32 silver badges33 bronze badges

2 Comments

paxdiablo Over a year ago

No, actually it will expand the $1 okay (to possibly an empty string) and leave the * untouched, in which case you'll get an error, unless you actually have a literal *.html file which is both unlikely and unwise :-)

James Polley Over a year ago

But as @paxdiablo also said, the actual root problem here is almost certainly that it's hanging on the second sed. Using bash -x is a handy troubleshooting technique though.

Collectives™ on Stack Overflow

sed command doesn't work in shell script

3 Answers 3

2 Comments

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related