how to replace a variable in shell script string

Question

I'm having problems getting this to work...

I have a variable that is holding a SQL to with a placeholder:

echo $SQL
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('$BATCH_END')

I have another variable that holds the value:

echo $BATCH_END
2012-11-14 17:06:13

I want to replace the placeholder with the value. I'm not particularly great at Unix scripting, but I've tried this:

echo $SQL | sed -e "s/'$BATCH_END/$BATCH_END/g"

but it still doesn't get replaced...

Can anyone help? I want to replace the placeholder, and keep the final string assigned to $SQL

I also need to know how to get the value of the output back into the variable, for example, I tried:

 SQL=`echo "$SQL" | echo "${SQL//\$BATCH_END/$BATCH_END}"`

Too much echo. Use SQL="${SQL//\$BATCH_END/$BATCH_END}" instead — doubleDown
– doubleDown, Commented Nov 14, 2012 at 7:49
Here is a more general version of this question, covering replacement of all variables without knowing in advance what the variables will be named. — Mark Haferkamp
– Mark Haferkamp, Commented Jun 7, 2015 at 9:49

sampson-chen · Accepted Answer · 2012-11-14 08:03:24Z

49

You are missing the end of that single-quote pair in your script.

Change from:

echo $SQL | sed -e "s/'$BATCH_END/$BATCH_END/g"

To:

echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g"

Updated - as per followup comment:

To save the result of the above replacement back into $SQL, do either of the following:

# Preferred way
SQL=$(echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g")

# Old way
SQL=`echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g"`

This is called command substitution. Either syntax ($(...) vs. enclosure by backticks) works, but the preferred one allows you to do nesting.

The preferred-preferred way: Herestring

This is probably a bit more advanced than what you care about, but doing it in the following way will save you a subprocess from having to use echo unnecessarily:

SQL=$(sed -e "s/\$BATCH_END/$BATCH_END/g" <<< $SQL)

edited Nov 14, 2012 at 8:03

answered Nov 14, 2012 at 6:25

sampson-chen

47.6k13 gold badges87 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Jonathan Leffler Over a year ago

The replacement value still needs quotes around it.

Richard G Over a year ago

ok I got this far:SQL=echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g", but the assignment back is not working although the SED seems to be now. How do I assign it back to the SQL variable?

sampson-chen Over a year ago

@RichardG See my updated answer for assigning the replaced result back into the SQL variable, I think that is what you are looking for.

gniourf_gniourf · Accepted Answer · 2012-11-14 06:32:00Z

3

In my terminal:

$ SQL="SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('\$BATCH_END')"
$ # (observe: I escaped the $ sign to have the same variable as you)
$ echo "$SQL"
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('$BATCH_END')
$ BATCH_END="2012-11-14 17:06:13"
$ echo "$BATCH_END"
2012-11-14 17:06:13
$ # Now the replacement:
$ echo "${SQL//\$BATCH_END/$BATCH_END}"
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('2012-11-14 17:06:13')

Done!

answered Nov 14, 2012 at 6:32

gniourf_gniourf

47.4k10 gold badges105 silver badges113 bronze badges

4 Comments

Richard G Over a year ago

ok that's a neat approach rather than SED, but how do I get it assigned back to the SQL variable, tried: SQL=echo "${SQL//\$BATCH_END/$BATCH_END}"

Jonathan Leffler Over a year ago

You should be able to use: SQL=${SQL//\$BATCH_END/$BATCH_END} just like that. You might need double quotes around the RHS of the assigment in some shells, but not in bash (and only bash, AFAIK, supports the // modifier on parameter expansion — though it wouldn't surprise me to find that zsh does too).

Bruno Bieri Over a year ago

Great listing all used commands! Helped to understand what's going on.

sjd Over a year ago

how can the same be achieved in ksh. the same script says bad substitution

kmkaplan · Accepted Answer · 2012-11-14 06:56:08Z

1

You need to quote the first $ so that it does not get expanded as a shell variable.

echo "$SQL" | sed -e "s/'\$BATCH_END'/'$BATCH_END'/g"

… Or choose an easier placeholder, something like @BATCH_END@ for instance.

To assign the result back to $SQL you will need some more shell escaping:

SQL=`echo "$SQL" | sed -e "s/'\\\$BATCH_END'/'$BATCH_END'/g"`

edited Nov 14, 2012 at 6:56

answered Nov 14, 2012 at 6:32

kmkaplan

19k4 gold badges55 silver badges65 bronze badges

1 Comment

Jonathan Leffler Over a year ago

Strongly recommend SQL=$(...) over SQL=`...` .

Jonathan Leffler · Accepted Answer · 2012-11-14 06:32:07Z

One way to do it is with 'differential quoting' in a single argument:

echo "$SQL" | sed -e 's/$BATCH_END/'"$BATCH_END/g"

The first part of the -e option is in single quotes, so the shell does not expand the first $BATCH_END and it can match the word in the SQL statement. The second part is in double quotes, so the shell expands the second $BATCH_END and places its text into the SQL.

If you needed to worry about the single quotes around $BATCH_END, you'd have to play other tricks; probably a backslash would be simplest (and it's a viable option anyway):

echo "$SQL" | sed -e "s/'\$BATCH_END'/'$BATCH_END'/g"

The backslash stops the shell expanding the first $BATCH_END but the absence of a backslash means the second is expanded. Inside the double quotes, the single quotes lose their 'no expansion' property.

Brian Campbell · Accepted Answer · 2012-11-14 07:08:15Z

0

The problem is that you are using a double-quoted string in the shell. In a double quoted string, variables like $BATCH_END get interpreted as shell variables and interpolated. The ' character has no special meaning within a double quoted string; it doesn't prevent variables from being interpolated. So your $BATCH_END string is being substituted in both places; your sed invocation is equivalent to:

sed -e "s/'2012-11-14 17:06:13/2012-11-14 17:06:13/"

Which as you can see, is not very helpful (you also have a stray ' in there). You need to escape the $ sign, to prevent it from being interpreted as a shell variable:

sed -e "s/\$BATCH_END/$BATCH_END/"

edited Nov 14, 2012 at 7:08

answered Nov 14, 2012 at 6:32

Brian Campbell

335k59 gold badges367 silver badges344 bronze badges

Comments

dmigo · Accepted Answer · 2020-11-10 10:37:25Z

0

Situation to add 80% memory in "maxmemory" in redis.conf:

# GET TOTAL MEMORY
totalmemory=$(awk '{ printf "%.2f", $2/1024 ; exit}' /proc/meminfo | awk -F"." '{print $1}');

# CALCUL
ramredis=$(echo $totalmemory/100*80 | bc);

# APPLY
sed -i 's/#maxmemory/maxmemory x mb/g' /etc/redis.conf;sed -i 's/ x /'\ $ramredis'/g' /etc/redis.conf;

edited Nov 10, 2020 at 10:37

dmigo

3,0555 gold badges47 silver badges66 bronze badges

answered Nov 9, 2020 at 21:02

Thiago Silva

211 bronze badge

Comments

Russo · Accepted Answer · 2022-01-20 11:06:03Z

0

Use double quote

sed -i "s/targetString/$newName/g" targetFile.toml

answered Jan 20, 2022 at 11:06

Russo

3,2404 gold badges41 silver badges53 bronze badges

Collectives™ on Stack Overflow

how to replace a variable in shell script string

7 Answers 7

3 Comments

4 Comments

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

7 Answers 7

3 Comments

4 Comments

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related