Bash variable substitution and strings

Question

Let's say I have two variables:

a="AAA"
b="BBB"

I read a string from a file. This string is the following:

str='$a $b'

How to create a new string from the first one that substitutes the variables?

newstr="AAA BBB"

Community · Accepted Answer · 2017-05-23 12:06:52Z

6

bash variable indirection whithout `eval`:

Well, as eval is evil, we may try to make this whithout them, by using indirection in variable names.

 a="AAA"
 b="BBB"
 str='$a $b'

 newstr=()
 for cnt in $str ;do
     [ "${cnt:0:1}" == '$' ] && cnt=${cnt:1} && cnt=${!cnt}
     newstr+=($cnt)
   done
 newstr="${newstr[*]}"

 echo $newstr
 AAA BBB

Another try:

var1="Hello"
var2="2015"

str='$var1 world! Happy new year $var2'

newstr=()
for cnt in $str ;do
    [ "${cnt:0:1}" == '$' ] && cnt=${cnt:1} && cnt=${!cnt}
    newstr+=($cnt)
  done
newstr="${newstr[*]}"

echo $newstr 
Hello world! Happy new year 2015

Addendum As correctly pointed by @EtanReisner's comment, if your string do contain some * or other glob expendable stings, you may have to use set -f to prevent bad things:

cd /bin
var1="Hello"
var2="star"
var3="*"
str='$var1 this string contain a $var2 as $var3 *'

newstr=()
for cnt in $str ;do
     [ "${cnt:0:1}" == '$' ] && cnt=${cnt:1} && cnt=${!cnt};
     newstr+=("$cnt");
   done;
newstr="${newstr[*]}"

echo "$newstr"
Hello this string contain a star as * bash bunzip2 busybox....zmore znew

echo ${#newstr}
1239

Note: I've added " at newstr+=("$cnt"); to prevent glob expansion, but set -f seem required...

newstr=()
set -f
for cnt in $str ;do
    [ "${cnt:0:1}" == '$' ] && cnt=${cnt:1} && cnt=${!cnt}
    newstr+=("$cnt")
  done
set +f
newstr="${newstr[*]}"

echo "$newstr"
Hello this string contain a star as * *

Nota 2: This is far away from a perfect solution. For sample if string do contain ponctuation, this won't work again... Example:

str='$var1, this string contain a $var2 as $var3: *'

with same variables as previous run will render: ' this string contain a star as *' because ${!var1,} and ${!var3:} don't exist.

... and if `$str` do contain special chars:

As @godblessfq asked:

If str contains a line break, how do I do the substitution and preserve the newline in the output?

So this is not robust as every indirected variable must be first, last or space separated from all special chars!

str=$'$var1 world!\n... 2nd line...'
var1=Hello
newstr=()
set -f
IFS=' ' read -d$'\377' -ra array <<<"$str"
for cnt in "${array[@]}";do
    [ "${cnt:0:1}" == '$' ] && cnt=${cnt:1} && cnt=${!cnt}
    newstr+=("$cnt")
  done
set +f
newstr="${newstr[*]}"

echo "$newstr"
Hello world!
... 2nd line...

As <<< inline string add a trailing newline, last echo command could be written:

echo "${newstr%$'\n'}"

edited May 23, 2017 at 12:06

CommunityBot

11 silver badge

answered Jan 7, 2015 at 10:20

F. Hauri - Give Up GitHub

72.9k19 gold badges136 silver badges153 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

choroba Over a year ago

You may later need to implement things like ${var} etc., but nice.

fedorqui Over a year ago

This is a very nice way to handle it by using the ${!var} indirection. Well done!

Etan Reisner Over a year ago

This expands globs if they occur in the string. You'd want to use set -f to prevent that. It also acts like tr -s ' ' on the input string which may be important. It also is assigning the full string to n[0] and not changing n from an array to a string variable (though this doesn't really matter at all).

F. Hauri - Give Up GitHub Over a year ago

@EtanReisner You'r right, but there is a lot of other bugs, like the fact that source string can't have ponctuation in it: $var1: $var=$var3 won't work... This have to be adapted in regard to specific requirements... Anyway, I've added a note about set -f.

Etan Reisner Over a year ago

Oh, sure that was far from comprehensive advice. It just pointed out a few of the problems with this solution. You pointed some other quite relevant ones. But those are "algorithm" details and not implementation ones. And yes, you need both set -f and the quotes in ("$cnt") to prevent pathname expansion at both points.

|

Roly · Accepted Answer · 2022-04-26 11:47:02Z

5

The easiest solution is to use eval:

eval echo "$str"

To assign it to a variable, use command substitution:

replaced=$(eval echo "$str")

edited Apr 26, 2022 at 11:47

Roly

2,2403 gold badges21 silver badges36 bronze badges

answered Jan 7, 2015 at 10:01

choroba

245k27 gold badges221 silver badges304 bronze badges

3 Comments

mimipc Over a year ago

This is perfect. But I think it could be risky from a security standpoint so I'll be careful.

choroba Over a year ago

@mimipc: If you control $str, nothing bad should happen even if $a and $b were wild.

fedorqui Over a year ago

Interesting reading: Eval command and security issues

PBS · Accepted Answer · 2021-11-28 07:16:07Z

0

Disclaimer: I only discovered perl an hour ago. But this seems to work robustly, whatever special characters you throw at it:

newstr=$(a2="$a" b2="$b" perl -pe 's/\$a\b/$ENV{a2}/g; s/\$b\b/$ENV{b2}/g' <(echo -e "$str"))

Test:

a='A*A\nA'
b='B*B\nB'
str='$a $aa * \n $b $bb'

newstr=$(a2="$a" b2="$b" perl -pe 's/\$a\b/$ENV{a2}/g; s/\$b\b/$ENV{b2}/g' <(echo -e "$str"))

echo -e "$newstr"

Output:

A*A
A $aa * 
 B*B
B $bb

answered Nov 28, 2021 at 7:16

PBS

1,42412 silver badges20 bronze badges

Comments

Fedor · Accepted Answer · 2023-02-14 14:34:18Z

0

I'd use awk solution with awk-variables. This will allow passing a text containing special chars and subsitute any placeholder with it. a workaround to recognize $ would be using [\x24]:

awk -v a="$a" -v b="$b" '{gsub("[\x24]a",a);gsub("[\x24]b",b); print}' <<< $str

here

-v defines variable a="$a"
[x24] is ASCII for $, so [x24]a equal to $a
gsub(x,y) - replaces x with y

answered Feb 14, 2023 at 14:34

Fedor

1167 bronze badges

Collectives™ on Stack Overflow

Bash variable substitution and strings

4 Answers 4

bash variable indirection whithout `eval`:

... and if `$str` do contain special chars:

9 Comments

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

bash variable indirection whithout eval:

... and if $str do contain special chars:

9 Comments

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related

bash variable indirection whithout `eval`:

... and if `$str` do contain special chars: