Using variable Names in For Loop (Bash)

Question

I have two sets of directories. In the first directory, I have 8 files as such:

file1.txt file2.txt file3.txt ... file8.txt

In the second directory called output, I have multiple subdirectories. I want to write for loop that goes into each sub-directory inside output (so they are directory1, directory2 etc...) and reads a file called database_file_#.txt. Currently, I have written this loop below, but it only works for one directory at a time.

for file in *; do 
name="${file%%.*}"
python script.py --parameter $file --out ../../output/directory1/${cts_name} 
--readin ../../output/directory1/database_file_1.txt 
done

For the second directory, I do the following:

 for file in *; do 
name="${file%%.*}"
python script.py --parameter $file --out ../../output/directory1/${cts_name} 
--readin ../../output/directory2/database_file_2.txt 
done

I do not want to keep re-writing directory# for all the directories inside output. Is there a way I can use a variable instead?

Write a function that takes the number as an argument. Call it 8 times. — William Pursell
– William Pursell, Commented Dec 17, 2018 at 22:31
What does this question actually have to do with "using variable names"? — Charles Duffy
– Charles Duffy, Commented Dec 17, 2018 at 23:32

Charles Duffy · Accepted Answer · 2018-12-18 15:53:08Z

2

for i in 1 2 3 4 5 6 7 8; do 
  for file in *; do 
    name="${file%%.*}"
    python script.py --parameter "$file" --out ../../output/directory"${i}/${cts_name}" \
        --readin "../../output/directory${i}/database_file_${i}.txt" 
  done
done

edited Dec 18, 2018 at 15:53

Charles Duffy

299k43 gold badges441 silver badges496 bronze badges

answered Dec 17, 2018 at 22:40

William Pursell

214k49 gold badges279 silver badges317 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

William Pursell Over a year ago

@CharlesDuffy Yeah, I guess that's more reasonable. :)

paulsm4 · Accepted Answer · 2018-12-17 22:43:58Z

0

It sounds like the key variable here is "#", not "file".

SUGGESTION:

for i in 1 2 3 4 5 6 7 8; do 
  filename=file$i.txt
  outdir=../../output/directory$i
  python script.py --parameter $filename --out $outdir/${cts_name} --readin $outdir/database_file_$i.txt 
done

PS: This assumes there's a 1::1 relationship between input file and #.

Otherwise, if there are N files for every #, you'll need a nested loop.

In any case, keep it as SIMPLE as possible (KISS).

edited Dec 17, 2018 at 22:43

answered Dec 17, 2018 at 22:30

paulsm4

123k23 gold badges175 silver badges245 bronze badges

1 Comment

Charles Duffy Over a year ago

Need more quotes if you don't want surprising/unexpected/undefined behavior here. --parameter "$filename" --out "$outdir/$cts_name" --readin "$outdir/database_file_$i.txt", f/e. The POSIX shell language family is a bit unfortunate insofar as what looks simple (in terms of minimizing the amount of syntax) and what is simple (in terms of minimizing the amount of undesired runtime behavior) are often very different.

wchmb · Accepted Answer · 2018-12-18 00:48:37Z

0

On a first attempt you can try this (but continue reading):

for f in $(ls -R); do
  if [[ $f =~ database_file_+([0-9])\.txt ]]; then
    python script.py ... --readin "$f"
  fi
done

Where +([0-9]) matches one or more numbers

Thanks to Charles Duffy comment, this snippet is much more accurate:

find . -type f -regex ".*/database_file_[0-9]+\.txt$" -exec python script.py ... --readin {} \;

edited Dec 18, 2018 at 0:48

answered Dec 17, 2018 at 23:08

wchmb

1,3632 gold badges20 silver badges27 bronze badges

4 Comments

Charles Duffy Over a year ago

Why you shouldn't parse the output of ls -- one can do the same search much more robustly with find, without having undefined behavior in the presence of filenames with nonprintable characters. (Also, not all versions of ls -R emit fully-qualified paths -- some will emit dirname: and then file, not dirname/file, making that output utterly useless if you need the actual path to the file that was found)

tshiono Over a year ago

In addition to the comment by Charles Duffy, your regex is incorrect. You may want to say database_file_[0-9]+\.txt instead. Another correction is you should enclose $f with double quotes.

wchmb Over a year ago

@tshiono you can omit the double quotes for $f within double brackets because of this: unix.stackexchange.com/questions/68694/…

Charles Duffy Over a year ago

Note too that =~ enables ERE syntax (where matching one-or-more digits looks like [[:digit:]]+), not extglob syntax. It would be [[ $f = ... ]] for ... to be parsed as a glob.

Collectives™ on Stack Overflow

Using variable Names in For Loop (Bash)

3 Answers 3

1 Comment

1 Comment

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related