Unix for loop skipping one value in script, works properly in command line

Question

I am running the following commands in command line:

for DATAFILE in `find dir_name -type f -mtime +10 | egrep -v -e 'archive/'`
do
    echo 'Data file name- ' "$DATAFILE"
    echo 'Base name ' 
    BASENAME=`basename "${DATAFILE}"`
    DESTFILE="${BASENAME}"_`date +"%Y%m%d%H%M%S"`
    echo "Dest file - "$DESTFILE
done

I get the following result for this:

Data file name-  DIR_PATH_1/file_1.txt
Base name
Dest file - file_1.txt_20120719041239
Data file name-  DIR_PATH_2/file_2.txt
Base name
Dest file - file_2.txt_20120719041239

When I put the same commands in a shell script and execute, I get the following result:

Data file name-  DIR_PATH_1/file_1.txt
DIR_PATH_2/file_2.txt
Base name
Dest file - file_2.txt_20120719040956

I have checked the script for Control-M and other junk characters. Also, I don't have any extra steps in the shell script (no parameters and all).

Can someone point me in the right direction.

Thanks.

Update 1:

I made the following change to the loop:

Earlier:

for DATAFILE in `find ${ROOT_DIR} -type f -mtime +$DAYS_ARCH | 
                     egrep -v -e 'archive/'`

Now:

find ${ROOT_DIR} -type f -mtime +$DAYS_ARCH |
     egrep -v -e 'archive/' | while read DATAFILE

It seems to be working properly now. I am still testing to confirm this.

Update 2:

Changing from FOR to WHILE loop has fixed the issue. But still I am not able to understand why this is happening. Anyone?

To add, the users running the shell script and from the command line are different. But I have given '777' to all files and directories. — visakh
– visakh, Commented Jul 19, 2012 at 8:18
Your for loop appears to be treating the whole output of find as a single field. What else do you have in your script? Has the IFS var been modified? — Shawn Chin
– Shawn Chin, Commented Jul 19, 2012 at 8:25
What shell is being used for the script? It looks like some feature has been turned on in the script that isn't turned on at the CLI. I regularly get these issues as I use zsh as by day-to-day shell, but write most scripts in bash as it's more likely to be installed — Anya Shenanigans
– Anya Shenanigans, Commented Jul 19, 2012 at 8:29
As @Petesh says: I'd start by putting #!/bin/sh on top of the script. (must be the first line!) — wildplasser
– wildplasser, Commented Jul 19, 2012 at 8:45

Henk Langeveld · Accepted Answer · 2012-08-10 00:04:45Z

1

Capturing find output in backticks will effectively join all lines. This will break file names with embedded spaces.

for DATAFILE in find ${ROOT_DIR} -type f -mtime +$DAYS_ARCH | egrep -v -e 'archive/'

The while read ... loop will read exactly one pathname at a time, even if they contain white space or other special characters.

answered Aug 10, 2012 at 0:04

Henk Langeveld

8,5461 gold badge46 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Michał Šrajer · Accepted Answer · 2012-08-09 07:40:29Z

1

Add to your script:

echo "My shell is: " $0
echo "My IFS is:" $IFS

and compare it with results from interactive shell.

Make sure your script is executed by script you want, by adding a hashbang line.

According to man sh, IFS is defined as:

Input Field Separators. This is normally set to ⟨space⟩, ⟨tab⟩, and ⟨newline⟩. See the White Space Splitting section for more details.

answered Aug 9, 2012 at 7:40

Michał Šrajer

31.5k7 gold badges65 silver badges88 bronze badges

Collectives™ on Stack Overflow

Unix for loop skipping one value in script, works properly in command line

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related