0

I want to replace a string in a file with the value of a variable. I a string lvl in the template file prm.prm which needs to be replaced by the value of SLURM_ARRAY.

I tried using

sed -i 's/lvl/${SLURM_ARRAY}/' prm.prm

This replaces the string lvl with ${SLURM_ARRAY} and not its value. How can I rectify this?

5
  • 3
    Variables won't expand in single quotes. Use double quotes. Add g to replace multiple occurrences in the same line (s/pat/rep/g). Commented Nov 26, 2021 at 7:46
  • 2
    Note that there are pitfalls in using a variable for a sed substitute replacement. For example if it contains strings like / or &. Commented Nov 26, 2021 at 7:51
  • Is SLURM_ARRAY an array? What's "the value of" an array??? Commented Nov 26, 2021 at 20:29
  • @dan Is there a way to pass a value to sed (e.g. through the env) that could be used in the replacement literal? Commented Nov 26, 2021 at 20:32
  • @ikegami No. The closest thing to this is maybe the r command. rfile prints the contents of file after the matching address, with no extra interpolation. However it can be done with a perl one liner. I will post an answer. Commented Nov 27, 2021 at 14:55

2 Answers 2

1

Every character between single quotes is used literally.

You could use double quotes instead as follows:

sed -i "s/lvl/${SLURM_ARRAY}/" prm.prm

However, your code now suffers from a code injection bug. There are characters (e.g. /) that will cause problems if found in the value of SLURM_ARRAY. To avoid this, these characters needs to be escaped.

quotemeta() { printf %s "$1" | sed 's/\([^a-zA-Z0-9]\)/\\\1/g'; }

sed -i "s/lvl/$( quotemeta "$SLURM_ARRAY" )/" prm.prm

However, it would be best to avoid generating programs from the shell. But that would require avoiding sed since it doesn't provide the necessary tools. For example, a couple of Perl solutions:

perl -i -spe's/lvl/$s/' -- -s="$SLURM_ARRAY" prm.prm
S="$SLURM_ARRAY" perl -i -pe's/lvl/$ENV{S}/' prm.prm
Sign up to request clarification or add additional context in comments.

Comments

0

Replace pattern with the value of an environment variable, with no extra interpolation of the contents:

Using perl:

export SLURM_ARRAY
perl -pe's/lvl/$ENV{SLURM_ARRAY}/g' prm.prm

Using awk:

export SLURM_ARRAY

awk '
{
    if (match($0, /lvl/)) {
        printf "%s", substr($0, 1, RSTART - 1)
        printf "%s", ENVIRON["SLURM_ARRAY"]
        print substr($0, RSTART + RLENGTH)
    }
    else {
        print
    }
}
' prm.prm

There's also SLURM_ARRAY=$SLURM_ARRAY perl ...etc or similar, to set the environment of a single process.

It can also be done with the variable as an argument. With both perl and awk you can access and modify the ARGV array. For awk you have to reset it so it's not processed as a file. The perl version looks like perl -e 'my $r = $ARGV[0]; while (<STDIN>) {s/lvl/$r/g; print}' "$SLURM_ARRAY" < prm.prm. It looks even better as perl -spe's/lvl/$r/g' -- -r="$SLURM_ARRAY". Thanks to ikegami.

For awk, I should say that the reason for not using awk -v r=foo is the expansion of C escapes. You could also read the value from a file (or bash process substitution).

3 Comments

Actually, what I needed was the one you suggested in the comments above - changing of quotes and everything works perfect.
@rnp yeah I didn't realise that it was someone else asking questions in the comments.
@ikegami Thanks. -p was a typo from copy pasting. I didn't know about -s, that's a big improvement. I've added it to the answer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.