Shell script while read loop executes only once

Question

I am writing a shell script to batch process .mov files off my camera through Handbrake to save HD space. The script searches a directory with 'find' and then runs Handbrake on each .mov file found, matching the creation date of the resulting file to the source file's date with 'touch'.

I originally did this with a for loop:

for i in $(find "$*" -iname '*.mov') ; do
  ~/Unix/HandbrakeCLI --input "$i" --output "$i".mp4 --preset="Normal"
  touch -r "$i" "$i".mp4
done

This worked, but failed if the input files had spaces in their file names. So I tried a while loop instead:

find "$*" -iname '*.mov' | while read i ; do
  ~/Unix/HandbrakeCLI --input "$i" --output "$i".mp4 --preset="Normal"
  touch -r "$i" "$i".mp4
done

The problem with this loop is that it works for the first file in the directory, and then exits the loop. Note that if I substitute an "echo $i" in the body of the while loop, it prints all of the .mov files in the directory, so the loop is structured correctly.

I believe there is a partial answer to my question on this stackoverflow thread. But the solution is specific to ssh and doesn't solve the general problem. Seems to have something do do with stdin being used by a sub-process, but I don't exactly understand how this works.

Any advice?

I'm on OSX 10.6

This can't be answered without knowing what $* is supposed to be. Is it supposed to be a list of files or directories? Or the current directory? Looks very much like you just use the wrong file argument with the wrong shell special parameter (e.g. use file "$@" instead of "$*"). — Jens
– Jens, Commented Aug 18, 2011 at 21:15
"$*" is quite certainly wrong; you definitely want "$@" instead. — Charles Duffy
– Charles Duffy, Commented Jul 1, 2013 at 21:53

Community · Accepted Answer · 2017-05-23 12:16:58Z

3

Taken from this answer: I now echo nothing into HandbrakeCLI to ensure it's not using the same stdin as my script:

find . -name "*.mkv" | while read FILE
do
    echo "" | handbrake-touch "$FILE"

    if [ $? != 0 ]
    then
        echo "$FILE had problems" >> errors.log  
    fi
done

...and it works as intended/expected.

edited May 23, 2017 at 12:16

CommunityBot

11 silver badge

answered Jul 1, 2013 at 21:47

Rodrigo Polo

4,9392 gold badges31 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

l0b0 · Accepted Answer · 2013-07-09 08:29:06Z

2

One possibility is to use the safe find:

while IFS= read -r -d '' -u 9
do
    ~/Unix/HandbrakeCLI --input "$REPLY" --output "$REPLY".mp4 --preset="Normal"
    touch -r "$REPLY" "$REPLY".mp4
done 9< <( find "$@" -type f -print0 )

This should be POSIX compatible, but only works if neither HandbrakeCLI nor touch read from standard input and no file names contain newlines:

find "$@" -type f -print0 | while IFS= read -r
do
    ~/Unix/HandbrakeCLI --input "$REPLY" --output "$REPLY".mp4 --preset="Normal"
    touch -r "$REPLY" "$REPLY".mp4
done

edited Jul 9, 2013 at 8:29

answered Apr 6, 2011 at 12:27

l0b0

59.6k32 gold badges155 silver badges247 bronze badges

4 Comments

beibei2 Over a year ago

I tried running this and got an error: "syntax error near unexpected token `<' I should note that I don't understand what the first two lines mean, so I'm not able to fix it myself.

sehe Over a year ago

that means that you are probably not on bash. Please report output of 'echo $0' in the context of the script? Or, what version of bash is this (bash --version)

beibei2 Over a year ago

bash version: GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin10.0) echo $0: /Volumes/Macintosh HD/Users/N/Unix/bin/nsqueezemovs

Charles Duffy Over a year ago

@beibei Does your script start with #!/bin/sh or #!/bin/bash? It needs to be the latter for <() to work.

sehe · Accepted Answer · 2011-04-05 09:21:50Z

1

You are probably running with a shellopt '-e' (exit on errors)

Try

set +e

answered Apr 5, 2011 at 9:21

sehe

400k49 gold badges475 silver badges673 bronze badges

3 Comments

beibei2 Over a year ago

Setting +e did not make any difference. Handbrake executes once on the first file in the while-read loop and then breaks out of the loop and executes the remainder of the script. I don't thing the problem has to do with errors, because handbrake completes successfully.

sehe Over a year ago

Could you try to do the Handbrake step in a subshell (Handbrake)

beibei2 Over a year ago

I tried running handbrake in a subshell, but it had no effect on the problem I'm having. I tried subshelling the entire loop, and also just the call to handbrake. Same result both ways.

Community · Accepted Answer · 2017-05-23 11:53:21Z

1

This link fixed the problem for me on Ubuntu Linux 11.04 .

Preventing a child process (HandbrakeCLI) from causing the parent script to exit

Here is the code that works for me:

ls -t *.mpeg | tail -n +2 | while read name
do echo "$name" ; in="$name" ; out=coded/"`echo $name | sed 's/.mpeg$/.mkv/'`"
echo "" | HandBrakeCLI -i "$in" -o "$out" -a '1,2' ; mv -v "$name" trash/"$name"
done

edited May 23, 2017 at 11:53

CommunityBot

11 silver badge

answered Aug 18, 2011 at 19:02

gabkdlly

1113 bronze badges

Comments

jpollock · Accepted Answer · 2012-10-07 22:08:31Z

I've got the same problem. I think HandbrakeCLI is consuming stdin. I don't have my actual system here, but my testing shows that's what is probably happening.

"output" is a file with the numbers 1-10, each on their own line.

while read line ; do echo "MAIN: $line"; ./consumer.sh; done < output

consumer.sh:

#!/bin/sh

while read line ; do
  echo "CONSUMER: $line"
done

Results:

MAIN: 1
CONSUMER: 2
CONSUMER: 3
CONSUMER: 4
CONSUMER: 5
CONSUMER: 6
CONSUMER: 7
CONSUMER: 8
CONSUMER: 9
CONSUMER: 10

Changing the outer while loop will fix the problem:

while read line ; do echo "MAIN: $line"; ./consumer.sh < /dev/null; done < output

Results:

MAIN: 1
MAIN: 2
MAIN: 3
MAIN: 4
MAIN: 5
MAIN: 6
MAIN: 7
MAIN: 8
MAIN: 9
MAIN: 10

Josh Cartwright · Accepted Answer · 2013-01-08 21:08:54Z

0

Consider just using find to invoke a shell, instead of wrapping find in a while loop.

find "$@" -iname '*.mov' -exec sh -c '
    ~/Unix/HandbrakeCLI --input "$1" --output "$1".mp4 --preset="Normal"
    touch -r "$1" "$1".mp4
' _ {} ';'

answered Jan 8, 2013 at 21:08

Josh Cartwright

8015 silver badges7 bronze badges

2 Comments

beibei2 Over a year ago

Thanks Josh. Any chance you could elaborate on the meaning of the final line?

Josh Cartwright Over a year ago

Sure! It is, in a clean way, passing arguments to the inline script. Consider sh -c 'echo "$1"' _ "hello,world". sh will run the basic, inline script echo "$1" passing as its arguments _ and hello,world. By convention, the 0th argument should hold the pathname to the command itself, which in this case doesn't make sense, so _ is just a useless placeholder. Hopefully that helps!

Collectives™ on Stack Overflow

Shell script while read loop executes only once

6 Answers 6

Comments

4 Comments

3 Comments

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

Comments

4 Comments

3 Comments

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related