Bash not evaluating variable when redirecting command output

Question

I want a shell script to process many input files. I am using variables for input names and renaming intermediate files. I am unable to redirect output to a filename.

The shell script is run like this:

./trim_pair_align_ab1.sh 116102 128861

The script can echo the variables correctly (lines 15 and 16) and they match expectation.

./temp/45739_1_PET22-001_116102_00_trim.fastq
./temp/45739_1_PET22-001_116102_00_trim.fasta

It can't evaluate the filename when the command redirects output

line 17: temp/$(echo $f|sed 's/_trim.fastq/_trim.fasta/'): No such file or directory

8 for f in *.fastq
9 do
10 seqtk trimfq -q 0.05 $f > ./temp/$(echo $f|sed 's/.fastq/_trim.fastq/')
11 done
12 
13 for f in `eval ls ./temp/*$1*_trim.fastq`;
14 do
15 echo $f
16 echo $(echo $f|sed 's/_trim.fastq/_trim.fasta/')
17 seqtk seq -A $f > ./temp/$(echo $f|sed 's/_trim.fastq/_trim.fasta/')
18 done

Why does the output redirect work on line 10 but not on line 17? What is making line 17 literal rather than evaluated?

You need to split lines of code so you can see what is happening. Do something like filename=./temp/$(echo $f|sed 's/_trim.fastq/_trim.fasta/') then echo "$filename". Thisfor f in eval ls ./temp/*$1*_trim.fastq; is a strange way of doing things, why do you need ls? Why not just for f in ./temp/*"$1"*_trim.fastq? — cdarke
– cdarke, Commented Jun 7, 2019 at 19:35
All the unquoted values (echo $f instead of echo "$f") are also sources of additional hard-to-predict behavior. See BashPitfalls #14. — Charles Duffy
– Charles Duffy, Commented Jun 7, 2019 at 20:06
...for line 17, it should be seqtk seq -A "$f" >"./temp/$(sed 's/_trim.fastq/_trim.fasta/' <<<"$f")" — Charles Duffy
– Charles Duffy, Commented Jun 7, 2019 at 20:07
...*if* you were still going to use sed at all, instead of going the much more efficient route and using a parameter expansion instead. — Charles Duffy
– Charles Duffy, Commented Jun 7, 2019 at 20:08
Line 13: Don't do that. Use for f in ./temp/*$1*_trim.fastq instead. — Dennis Williamson
– Dennis Williamson, Commented Jun 7, 2019 at 21:12

Charles Duffy · Accepted Answer · 2019-06-07 20:22:46Z

2

The command substitution is being expanded; the shell is simply printing the code you gave it, not the result of that expansion. For the real issue, you'll want to look further.

Take a close look at your echo showing the result of your command substitution:

./temp/45739_1_PET22-001_116102_00_trim.fasta

It already has a ./temp prefix.

When you add a second such prefix in your redirection (>./temp/$(...)), you're making your code try to create a file in ./temp/temp/. This directory doesn't exist, so you get an error.

That said, a better implementation would look like:

for f in ./temp/*"$1"*_trim.fastq; do
  seqtk seq -A "$f" >"${f%.fastq}.fasta"
done

answered Jun 7, 2019 at 20:22

community wiki

Charles Duffy

Sign up to request clarification or add additional context in comments.

1 Comment

landru27 Over a year ago

upvoted; @JoeT : to expand on this answer a bit, the original line 13 runs ls on each of a long list of individual files; you might have been expecting what happens when ls operates on a single directory, the result of which is a long list of files; these two things are similar, but different; notably, in the latter case, the directory name itself is not part of the output

Collectives™ on Stack Overflow

Bash not evaluating variable when redirecting command output

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related