Bash variable defaulting doesn't work if followed by pipe (bash bug?)

Question

I've just discovered a strange behaviour in bash that I don't understand. The expression

${variable:=default}

sets variable to the value default if it isn't already set. Consider the following examples:

#!/bin/bash
file ${foo:=$1}
echo "foo >$foo<"
file ${bar:=$1} | cat
echo "bar >$bar<"

The output is:

$ ./test myfile.txt
myfile.txt: ASCII text
foo >myfile.txt<
myfile.txt: ASCII text
bar ><

You will notice that the variable foo is assigned the value of $1 but the variable bar is not, even though the result of its defaulting is presented to the file command.

If you remove the innocuous pipe into cat from line 4 and re-run it, then it both foo and bar get set to the value of $1

Am I missing somehting here, or is this potentially a bash bug?

(GNU bash, version 4.3.30)

@ammoQ, nothing of the sort; documented behavior, and even part of the FAQ. — Charles Duffy
– Charles Duffy, Commented Oct 23, 2014 at 16:14

Cyrus · Accepted Answer · 2014-10-23 11:56:31Z

5

In second case file is a pipe member and runs as every pipe member in its own shell. When file with its subshell ends, $b with its new value from $1 no longer exists.

Workaround:

#!/bin/bash
file ${foo:=$1}
echo "foo >$foo<"

: "${bar:=$1}"     # Parameter Expansion before subshell

file $bar | cat
echo "bar >$bar<"

edited Oct 23, 2014 at 11:56

answered Oct 23, 2014 at 11:28

Cyrus

90.1k15 gold badges112 silver badges173 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

starfry Over a year ago

Does that mean that parameter expansion is performed in the subshell instead of before the subshell is created?

Cyrus Over a year ago

@starfry: chepner explained it properly in his answer.

chepner · Accepted Answer · 2014-10-23 16:03:13Z

2

It's not a bug. Parameter expansion happens when the command is evaluated, not parsed, but a command that is part of a pipeline is not evaluated until the new process has been started. Changing this, aside from likely breaking some existing code, would require extra level of expansion before evaluation occurs.

A hypothetical bash session:

> foo=5
> bar='$foo'
> echo "$bar"
$foo
# $bar expands to '$foo' before the subshell is created, but then `$foo` expands to 5
# during the "normal" round of parameter expansion.
> echo "$bar" | cat
5

To avoid that, bash would need some way of marking pieces of text that result from the new first round of pre-evaluation parameter expansion, so that they do not undergo a second round of evaluation. This type of bookkeeping would quickly lead to unmaintainable code as more corner cases are found to be handled. Far simpler is to just accept that parameter expansions will be deferred until after the subshell starts.

The other alternative is to allow each component to run in the current shell, something that is allowed by the POSIX standard, but is not required, either. bash made the choice long ago to execute each component in a subshell, and reversing that would break too much existing code that relies on the current behavior. (bash 4.2 did introduce the lastpipe option, allowing the last component of a pipeline to execute in the current shell if explicitly enabled.)

edited Oct 23, 2014 at 16:03

answered Oct 23, 2014 at 13:45

chepner

538k77 gold badges594 silver badges746 bronze badges

2 Comments

starfry Over a year ago

On my bash version 4.3.30, echo "$bar" | cat results in $foo being printed, not 5.

chepner Over a year ago

Yes; the example I showed is hypothetical, not representing any real version of bash, to demonstrate the extra work bash would need to do to avoid such double expansion if parameters were expanded in the current shell.

Collectives™ on Stack Overflow

Bash variable defaulting doesn't work if followed by pipe (bash bug?)

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related