0

I have a bash script that operates several similar find commands and reads the output into variables:

f_list=`find /foo -name bar | sort`
f_number=`find /foo -name bar | wc -l`
f_first=`find /foo -name bar | sort | head -n 1`

All this works as expected.

Assuming the "find" command to be expensive (time wise), is there a clever way of re-using one of the results for the others?

What I tried (and failed with) is:

f_list=`find /foo -name bar | sort`
f_number=`echo "$f_list" | wc -l`
f_first=`echo "$f_list" | head -n 1`

This doesn't work, but I hope it shows what I want to achieve.

It seems that putting the results into a variable spoils some format of the original output, which breaks stuff when sent again to the other commands.

Is there some clever way of achieving what I want?

EDIT

I created a fully working example you could recreate. In my working dir I have a folder "foo" with 3 files "bar1", "bar2", "bar3".

find_result=`find ./foo -type f -iname "bar*" | sort`
find_count1=`echo "$find_result" | wc -l`
echo "$find_result"
echo $find_count1
find_count2=`find ./foo -type f -iname "bar*" | wc -l`
find ./foo -type f -iname "bar*"
echo $find_count2

results in the expected

./foo/bar1
./foo/bar2
./foo/bar3
3
./foo/bar3
./foo/bar2
./foo/bar1
3

but when the result is empty (I modified the search criteria to find nothing)

find_result=`find ./foo -type f -iname "bor*" | sort`
find_count1=`echo "$find_result" | wc -l`
echo "$find_result"
echo $find_count1
find_count2=`find ./foo -type f -iname "bor*" | wc -l`
find ./foo -type f -iname "bor*"
echo $find_count2

the two results differ (notice the empty result line in front of the "1")

 
1
0

And thus I thought the culprint to be the extra line break in the echo command. Therefore I removed that (notice the "-n" in the second line):

find_result=`find ./foo -type f -iname "bor*" | sort`
find_count1=`echo -n "$find_result" | wc -l`
echo "$find_result"
echo $find_count1
find_count2=`find ./foo -type f -iname "bor*" | wc -l`
find ./foo -type f -iname "bor*"
echo $find_count2

which solves the problem for empty results

 
0
0

but now when there are results, the "wc -l" counts the wrong way so

find_result=`find ./foo -type f -iname "bar*" | sort`
find_count1=`echo -n "$find_result" | wc -l`
echo "$find_result"
echo $find_count1
find_count2=`find ./foo -type f -iname "bar*" | wc -l`
find ./foo -type f -iname "bar*"
echo $find_count2

yields in

./foo/bar1
./foo/bar2
./foo/bar3
2
./foo/bar3
./foo/bar2
./foo/bar1
3

So the problem is one line break that I thought must have an easy resolution to avoid being different between empty and non-empty find results. I have the feeling of missing something very simple.

Improve this question
4
  • 2
    This doesn't work Doesn't it? It looks like it should work. How do you know it doesn't work? But anyway, do not use ` backticks. Use $(..) instead. Commented Mar 4, 2021 at 14:12
  • "It modifies the output" is usually because of an unquoted expansion (see I just assigned a variable, but echo $variable shows something else), but that doesn't seem to be the case here. Commented Mar 4, 2021 at 14:18
  • @KamilCuk: I've tried repalcing the backticks and the result is exactly the same. Is there a reason why $(...) is better than `...`? Commented Mar 4, 2021 at 19:08
  • @Benjamin W.: I edited my question, adding a minimal example. I think there is no unquoted expansion or funny character involved. Commented Mar 4, 2021 at 19:09

3 Answers 3

Reset to default
2

Using a variable to reuse find's output should work. Maybe there are backslashes in your filenames that get interpreted by echo (although that should not happen by default). If so, then use printf %s "$f_list" instead. But without a complete verifiable example we cannot know for sure.

However, in this specific case you can also switch to the following command which is also safe if no file matches and safe in the very unusual case where some files have multi-line filenames.

shopt -s globstar dotglob nullglob
list=(/foo/**/bar)
number=${#list[@]}
first=${list[0]}
Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Notice that this differs from the find output in that it doesn't match hidden directories – you have to either use list=(/foo/{,.}**/bar) for that, or shopt -s dotglob.
Good catch. Thank you for the comment. I'd go with dotglob as /foo/{,.}**/bar matches /foo/bar and /foo/./bar at the same time.
That's the same path and would be normalized away, no? Only the first one would actually be in the expansion.
@BenjaminW. It is the same path and it will be normalized, but both versions will be in the expansion (because the there are actually two expansions). There is even another problem I didn't recognize so far: The parent directory of foo might also be included! You can try this using shopt -s globstar && mkdir -p a/{b,c}/c && echo a/b/{,.}**/c. This prints a/b/c a/b/./c a/b/../c.
Ah, if it has the same name! True. dotglob and nullglob is much cleaner then.
1

An alternative would be to use a temporary file. That would be something like:

temp=$(mktemp /tmp/prefixXXXXXXXXXXX)
find /foo -name bar | sort > $temp
f_number=`wc -l $temp`
f_first=`head -n 1 $temp`

rm -f $temp
Improve this answer

1 Comment

your solution works for me but I have to use for example wc -l <$temp instead of wc -l $temp otherwise wc would add unwanted extra output
0

Use an array to store the null delimited list returned by find -print0 | sort -z:

#!/usr/bin/env bash

# map null delimited output of command-group into the f_list array
mapfile -t f_list < <(find /foo -name bar -print0 | sort --zero-terminated)

# Number of entries in the f_list array
f_number=${#f_list[@]}

# First entry of the f_list array
f_first=${f_list[0]}
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.