Modifying a shell variable in a while loop does not work as expected

Question

This is a very simple bash script I wrote:

#!/bin/bash

ITEM_LIST=items.txt
LOG_FILE=log.log

TOTAL_ITEMS=$(wc -l ${ITEM_LIST} | awk '{ print $1 }')
let NOT_FOUND=0

cat ${ITEM_LIST} | while read item; do

    grep "${item}" ${LOG_FILE} > /dev/null
    FOUND=${?}
    if [ ${FOUND} -ne 0 ]; then
        let NOT_FOUND=NOT_FOUND+1
        echo "Item not found [${item}] Item not found number: ${NOT_FOUND}"
    fi

done

echo "Total items: ${TOTAL_ITEMS}"
echo "Total not found items: ${NOT_FOUND}"

I want to check if some items are present in a log file, count how many are not present, and print some kind of reporting (last two echo). At this moment, I'm running it from cygwin bash shell.

Consider these two sample files:

items.txt

first item
second item
third item
fourth item
fifth item

log.log

blahblah blah blah first item blah blah blah
second blah blah item
blah third item blah

Script's output:

[17:46:38]:/cygdrive/c/Temp/qpa# ./script2.sh 
Item not found [second item] Item not found number: 1
Item not found [fourth item] Item not found number: 2
Total items: 4
Total not found items: 0

Question:

Why is the script printing "Total not found items: 0"? It prints the current total on every echo on the loop (both NOT_FOUND variable).

Are there some bad practices in this shell script? Where and why?

Johannes Schaub - litb · Accepted Answer · 2009-04-30 16:17:57Z

The pipe you use here:

cat ${ITEM_LIST} | ...

Will execute the while loop afterwards in a sub-shell. But that means that the NOT_FOUND variable won't be updated in the parent shell, but only in the sub-shell you execute the loop in.

Include the echo commands at the end inside that sub-shell instance:

cat ${ITEM_LIST} | { 
  while read item; do
    grep "${item}" ${LOG_FILE} > /dev/null
    FOUND=${?}
    if [ ${FOUND} -ne 0 ]; then
        let NOT_FOUND=NOT_FOUND+1
        echo "Item not found [${item}] Item not found number: ${NOT_FOUND}"
    fi
  done

  echo "Total items: ${TOTAL_ITEMS}"
  echo "Total not found items: ${NOT_FOUND}"
}

That problem is also explained in a Bash FAQ item. Hope this helps.

As the FAQ explains, in that particular case, you can also rewrite it so it reads:

while read item; do
    grep "${item}" ${LOG_FILE} > /dev/null
    FOUND=${?}
    if [ ${FOUND} -ne 0 ]; then
        let NOT_FOUND=NOT_FOUND+1
        echo "Item not found [${item}] Item not found number: ${NOT_FOUND}"
    fi
done < ${ITEM_LIST}

Prefer the second option in this case, since it will get rid of one "useless use of cat" :)

lothar · Accepted Answer · 2009-04-30 16:13:54Z

0

Use expr to do the math

NOT_FOUND=0; NOT_FOUND=`expr ${NOT_FOUND} + 1`; echo ${NOT_FOUND}

1

@litb is right, you need to update NOT_FOUND in your main script, not in the sub shell spawned by the pipe command.

answered Apr 30, 2009 at 16:13

lothar

20.3k6 gold badges48 silver badges60 bronze badges

1 Comment

sourcerebels Over a year ago

thank you for your response, I tested @litb second solution and it works

Collectives™ on Stack Overflow

Modifying a shell variable in a while loop does not work as expected

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related