Using sed to replace string in file with contents of variable of that name

The problem here is that by the time the sed command is actually run (and therefore by the time it retrieves the variable-name), the sed command must have been fully assembled (including substituting the Bash variable's value into the replacement string); so everything happens in the wrong order.

Or, taking a higher-level view, the problem is that sed doesn't know about Bash variables, so you need Bash to provide the details of the variables, but Bash doesn't know about sed replacements, so it doesn't have any way of knowing what variables you need the details of.

The fix, as long as you want to use Bash variables, is to use more Bash: you need to identify the relevant variable-names before you first call sed. The below shows how you can do that.

To get the list of all variable-names in your file, you can write something like this:

grep -o '%[a-z_][a-z_]*%' FILE | grep -o '[a-z_][a-z_]*' | sort -u

(The first grep gets all expressions of the form %...%. The second grep filters out the percent-signs; or you can use sed for that, if you prefer. The sort -u eliminates the duplicates, since you only need the list of distinct variable-names.)

Armed with that, you can assemble a sed command that performs all the necessary replacements:

sed_args=()
while read varname ; do
    sed_args+=(-e "s/%$varname%/${!varname}/g")
done < <(grep -o '%[a-z_][a-z_]*%' FILE | grep -o '[a-z_][a-z_]*' | sort -u)
sed "${sed_args[@]}" FILE

(Note the use of ${!varname} to mean "take the value of $varname as a variable-name, and return the value of that variable." This is what §3.5.3 "Shell Parameter Expansion" of the Bash Reference Manual calls "indirect expansion".)

You can wrap this in a function:

function replace_bash_variables () {
    local file="$1"
    local sed_args=()
    local varname
    while read varname ; do
        sed_args+=(-e "s/%$varname%/${!varname}/g")
    done < <(grep -o '%[a-z_][a-z_]*%' "$file" | grep -o '[a-z_][a-z_]*' | sort -u)
    if [[ "${#sed_args[@]}" = 0 ]] ; then
        # if no variables to replace, just cat the file:
        cat -- "$file"
    else
        sed "${sed_args[@]}" -- "$file"
    fi
}

replace_bash_variables OLD_FILE > NEW_FILE

You can also adjust the above to do line-by-line processing, so that it doesn't need to read the file twice. (That gives you more flexibility, since reading the file twice means you have to pass in the actual file, and can't (say) apply this to the output of a pipeline.)

Use this:

sed -E "s/%(\w+)%/\$\1/g"

For example this:

echo "abcdef %variable% blah" | sed -E "s/%(\w+)%/\$\1/g"

prints:

abcdef $variable blah

Using awk you can do this

awk '{gsub(/%user_home%/,"${user_home}")}1' file
NameVirtualHost *:80

<VirtualHost *:80>
  ServerName %server_name%

  ErrorLog /var/log/apache2/${user_home}_webapp_error.log
  CustomLog /var/log/apache2/${user_home}_webapp.log common

  DocumentRoot /home/${user_home}/web_app/public
</VirtualHost>

This replace the %user_home% to the variable ${user_home}

try with 1 sed but still need previously to catch the "set" content to know variables name and value

#!/bin/ksh
# YourFilename contain the file name of your file to treat (here passed as 1st parameter to a script)
YourFileName=$1

(set | sed 's/.*/#V0r:&:r0V#/'; cat ${YourFileName}) | sed -n "
s/$/²/
H

$  {
   x
   s/^\(\n *\)*//
# also reset t flag
   t varxs

:varxs
   s/^#V0r:\([a-zA-Z0-9_]\{1,\}\)=\([^²]*\):r0V#²\(\n.*\)%\1%/#V0r:\1=\2:r0V#²\3\2/
   t varxs
: tmpb

# clean the line when no more occurance in text
#   s/^#V0r:\([a-zA-Z0-9_]\{1,\}\)=\([^²]*\):r0V#²\n//
   s/^[^²]*:r0V#²\n//

# and next
   t varxs


# clean the  marker
   s/²\(\n\)/\1/g
   s/²$//

# display the result
   p
   }
"

• limitation here due to the use of char "²" not escaped so if ² appear in the file, could be annoying (so change this char as marker or translate it in the file)
• #V0r: and :r0V# are marker also and could be changed without problem