How to handle no matching files when using ${f%.gz} syntax

Question

I'm developing a script to process and move away compressed files that get dropped into a certain folder.

The script works perfectly as long as what gets dropped into the folder is a compressed file. However, if the script executes when there are no compressed files to process the bash function ${f%.gz} gives unexpected results.

Here is the script, with an example of the problem case afterwards:

FILES="$INGEST_DIR/*.gz"

for f in $FILES
do
    JUSTFILENAME=${f##/*/}
    syslog -s -l n "Archiving \"$JUSTFILENAME\""
    UNZIPPEDPATH=${f%.gz}
    syslog -s -l n "Moving \"$UNZIPPEDPATH\""
    UNZIPPEDNAME=${UNZIPPEDPATH##/*/}
    syslog -s -l n "    to \"ASR_DIR/$UNZIPPEDNAME\""
    syslog -s -l n "gunzip-ing $f"
    gunzip $f
    mv "$UNZIPPEDPATH" "$ASR_DIR/$UNZIPPEDNAME"
done

Again, it works perfectly if there's at least one .gz file in the target directory.

If there aren't any .gz, but there are other files in the directory (which must be there for other reasons) $FILES contains the expanded $INGEST_DIR plus the /*.gz, like this:

INGEST_DIR=/path/to/foo
FILES="$INGEST_DIR/*.gz"
echo $FILES

will show

/path/to/foo/*.gz

That isn't especially bad except that

for f in $FILES
do
    UNZIPPEDPATH=${f%.gz}
    echo $UNZIPPEDPATH
done

yields

somefile.txt someotherfile.exe yetsomeotherfile.dat

So is there an elegant way to not iterate if there are no such compressed files to handle? My script is working as well as it is because I just learned about ${f##/*/} and ${f%.gz} from this SO question & answer, so I'm thinking there might be a better way than

FILES="$INGEST_DIR/*.gz"

to start things off... or something to do right away before heading into the for loop.

I agree with devnull that when it comes to files you should use power of find tool, as for ${foo%boo} syntax, if it don't find boo in your foo var it will return foo unchanged. That syntax just tries to cut off boo from the end of the foo string. — rook
– rook, Commented Aug 7, 2013 at 10:25

devnull · Accepted Answer · 2013-08-07 10:06:38Z

2

Use find for getting the list of files instead:

for f in $(find . -type f -name "*.gz"); do

done

For limiting the matches to the current directory, you could say:

find . -maxdepth 1 -type f -name "*.gz"

answered Aug 7, 2013 at 10:06

devnull

124k33 gold badges247 silver badges234 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

tobinjim Over a year ago

Thank you @devnull, I like building the find right into the loop handler because it's clear what you're doing.

konsolebox Over a year ago

@devnull I think you set IFS to $'\n' as well to prevent splitting of filenames with spaces. Or if in Bash just use <code>find . -type f -name "*.gz" | while IFS= read -r f; do ...; done</code>

devnull Over a year ago

@konsolebox You could quote the $(...) and that would take care of spaces in filenames.

konsolebox Over a year ago

@devnull do you mean "$(...)"? but that would only be parsed as one string.

devnull Over a year ago

@konsolebox Did you try the quoted version?

|

konsolebox · Accepted Answer · 2013-08-07 11:05:10Z

1

If you're using bash you could set nullglob before the loop:

shopt -s nullglob
for f in $FILES

---- Add ----

Another way to do it is by using while read and process substitution:

while IFS= read -r f; do
    ...
done < <(command)

Where command could be by the use of find or compgen, only that find could be more specific to files:

find -type f -name '*.gz'
compgen -G '*.gz'

If we're reading input within the loop, we could use other FDs when opening the file:

while IFS= read -ru 4 f; do
    ...
done 4< <(command)

edited Aug 7, 2013 at 11:05

answered Aug 7, 2013 at 10:05

konsolebox

76.3k13 gold badges110 silver badges114 bronze badges

3 Comments

tobinjim Over a year ago

Thank you konsolebox, that works nicely. I'm going to accept your answer to help with your rep :) even though I personally am going to use the answer @devnull gave since it will be easier for me to know what's going on when I revisit the script 6 months from now!

konsolebox Over a year ago

Welcome. I added more suggestions just in case they would be helpful as well.

tobinjim Over a year ago

They are helpful -- I'm still not fluent speaking in bash... in particular, compgen was new to me and looks handy. cyberciti.biz/open-source/command-line-hacks/… <-- nice article, even for us Mac OS X users ;)

Collectives™ on Stack Overflow

How to handle no matching files when using ${f%.gz} syntax

2 Answers 2

6 Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related