string formatting in bash

Question

I have the following code that runs in a shell script

foo=`seq 1 1 100`
for i in $foo; do
  echo "input$i\_now"
done

Here's my question: Under some conditions, the output prints input1_now whereas othertimes it prints input1\_now. I'm sure something is different, but I can't figure out what makes it print one way or the other. If my code is

for i in $foo; do
   echo "input$i_now"
done

I will always get input with the rest of the line being omitted.

I know I can use input${i}_now instead and have it print correctly every time, but I'm mostly interested in understanding why the output is different under seemingly the same conditions.

UPDATE:

In the following example, the first part correctly formats the variables and text such that the \_ is replaced as _. However, the last part required me to place variables in curly brackets in order to have them formatted correctly.

echo "Enter Simulation #: "
read sim

g.mapset results

for i in `seq 1 1 100`; do

file=sim$sim\_run$i\_sum
g.copy $file\@expSim$sim\_$i,$file

file=sim$sim\_run$i\_average
g.copy $file\@expSim$sim\_$i,$file

for year in `seq 2004 1 2006`; do   

    file=sim$sim\_$year\_run$i\_sum
    g.copy $file\@expSim$sim\_$i,$file

    file=sim$sim\_$year\_run$i\_average
    g.copy $file\@expSim$sim\_$i,$file

done

years="2004 2005 2006"
times=`seq -w 1 16 365`
runs=`seq 1 1 100`

for year in $years; do
for ptime in $times; do
    for i in $runs; do
        if [ $i -eq 1 ]; then
            g.copy  vect=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i},sim${sim}_pts_${year}_${ptime}
        fi
        if [ $i -gt  1 ]; then  
        v.patch input=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i} output=sim${sim}_pts_${year}_${ptime} -e -a --o
        fi
    done
done
done

It prints inputx\_now consistently for me.

Ja͢ck
– Ja͢ck

2013-04-16 04:56:55 +00:00
Commented Apr 16, 2013 at 4:56 — Ja͢ck
– Ja͢ck, Commented Apr 16, 2013 at 4:56

rob mayoff · Accepted Answer · 2013-04-17 02:54:44Z

11

Is _ supposed to be a placeholder that is sometimes a different character?

In bash, "input$i\_now" with an actual _ will always produce input1\_now. Inside double-quotes, bash only removes a \ when it is followed by a $, a `, a ", a \, or a newline. See “Double Quotes” in the Bash Reference Manual. This is the POSIX standard behavior; see “Double-Quotes” in Shell Command Language.

UPDATE

If you write "input$i_now", bash will just print input. It will not print input1 or input1_now. It does this because _ is a valid parameter name character, so bash thinks you are asking for the value of the i_now parameter. Unless you have set i_now to a non-empty string, bash will expand $i_now to the empty string, thus turning "input$i_now" into input.

UPDATE 2

Now that you have posted real code, we can see what's going on.

First of all, in the real code you posted, you never used double-quotes around a parameter expansion. This makes a difference.

Outside of double-quotes, a \ is always removed. See “Quote Removal” in the Bash Reference Manual. Hence input$i\_now (with no surrounding double-quotes) expands to input1_now.

However, as I explained in my first update, _ is a parameter name character. See “Name” in Shell Command Language. So when bash sees input$i_now, it takes i_now as the parameter name.

Whether or not you're using double-quotes, you must separate the parameter name from the following character, if bash would otherwise treat the following character as part of the parameter name. You can do this by putting \ after the parameter name, or you can do it by putting the parameter name in {...}.

It is safer to always use {...}, because (as you have discovered?) \ is handled differently depending on whether it's inside double-quotes. If you go back and add double-quotes later, and you have used \, you will need to change the \ to {...} anyway.

UPDATE 3

Here is a demonstration of the effects of \, {...}, and double-quoting. First, we set up some variables:

$ year=2004 ptime=1 i=1 sim=123

Here's what happens with no quoting whatsoever:

$ echo vect=sim$sim_pts_$year_$ptime_run$i@expSim$sim_$i,sim$sim_pts_$year_$ptime
vect=sim1@expSim1,sim1

Here's what happens if we just use {...} without double-quotes:

$ echo vect=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i},sim${sim}_pts_${year}_${ptime}
vect=sim123_pts_2004_1_run1@expSim123_1,sim123_pts_2004_1

If we add double-quotes, they have no effect:

$ echo "vect=sim${sim}_pts_${year}_${ptime}_run${i}@expSim${sim}_${i},sim${sim}_pts_${year}_${ptime}"
vect=sim123_pts_2004_1_run1@expSim123_1,sim123_pts_2004_1

Here's what happens if we just use \:

$ echo vect=sim$sim\_pts\_$year\_$ptime\_run$i@expSim$sim\_$i,sim$sim\_pts\_$year\_$ptime
vect=sim123_pts_2004_1_run1@expSim123_1,sim123_pts_2004_1

Notice that each \ was removed. The shell removes a \ if it's not quoted.

If we add double-quotes, they prevent the shell from removing each \:

$ echo "vect=sim$sim\_pts\_$year\_$ptime\_run$i@expSim$sim\_$i,sim$sim\_pts\_$year\_$ptime"
vect=sim123\_pts\_2004\_1\_run1@expSim123\_1,sim123\_pts\_2004\_1

edited Apr 17, 2013 at 2:54

answered Apr 16, 2013 at 4:23

rob mayoff

387k69 gold badges842 silver badges887 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

ShaunLangley Over a year ago

However, it is not always removing the _ when preceeded by $i in the case of input$i_now

rob mayoff Over a year ago

Then you need to edit your post to include a runnable example that outputs input1_now instead of input1\_now. Right now your question is basically “My computer acts funny sometimes. Why? I can't provide any more details.”

ShaunLangley Over a year ago

If I write input$i_now it will output input1 and omit the rest of the line.

ShaunLangley Over a year ago

Yes. You are correct. it doesn't print input1 rather just input. But I'm still failing to understand why input$i\_now prints input1_now sometimes and other times input1\_now. It seems inconsistent. I'm sure I'm doing something different, but I can't see it.

ShaunLangley Over a year ago

@konsolebox nope. I'm editing my question to add the actual code

|

sastorsl · Accepted Answer · 2021-12-20 09:17:39Z

7

Underscore is a valid character in bash variable names so you need to protect the variable name from surrounding characters.

foo=`seq 1 1 100`
for i in $foo; do
  echo "input${i}_now"
done

The difference:

input$i_now    # "input" + $i_now
input${i}_now  # "input" + $i + "_now"
input"$i"_now  # input + "$i" + now
input$i\_now   # input + $1 + _now - \_ "escapes" the underscore

https://man7.org/linux/man-pages/man1/bash.1.html

name A word consisting only of alphanumeric characters and underscores, and beginning with an alphabetic character or an underscore. Also referred to as an identifier.

You have to protect the parameter name, either by PREFIX${braces}SUFFIX, with PREFIX"$quotes"SUFFIX or by escaping - \ - characters which are valid in parameter names. All "work" but I prefer braces which is also what the documentation refers to:

The parameter name or symbol to be expanded may be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.

edited Dec 20, 2021 at 9:17

answered Apr 16, 2013 at 7:14

sastorsl

2,1851 gold badge16 silver badges18 bronze badges

1 Comment

ShaunLangley Over a year ago

that's why I needed to use input$i\_now

Collectives™ on Stack Overflow

string formatting in bash

2 Answers 2

UPDATE

UPDATE 2

UPDATE 3

6 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

UPDATE

UPDATE 2

UPDATE 3

6 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related