568

I have this multi-line string (quotes included):

abc'asdf"
$(dont-execute-this)
foo"bar"''

How would I assign it to a variable using a heredoc in Bash?

I need to preserve newlines.

I don't want to escape the characters in the string. That would be annoying...

4
  • @JohnM - I have just tried a heredoc assignment with single-quoted 'EOF', with escaped linebreaks with ` in the content: if the second line has cd` command, I get back: ".sh: line X: cd: command not found"; but if I double-quote "EOF"; then bash variables ${A} do not get preserved as strings (they get expanded); but then, line-breaks are preserved - and, I don't have a problem running a command with cd in second line (and both 'EOF' and "EOF" seem to play well also with eval, for running a set of commands stored in a string variable). Cheers! Commented May 24, 2012 at 8:08
  • 1
    ... and to add to my previous comment: bash comments "#" in double-qouted "EOF" variable, if called via eval $VAR, will cause all of the rest of the script to be commented, as here $VAR will be seen as a single line; to be able to use bash # comments in multiline script, double-quote also variable in the eval call: eval "$VAR"`. Commented May 24, 2012 at 8:18
  • @sdaau: I had problems with eval ith this methods, but did not track it down since it was part of some package which evals some variables defined in it's config file. Error message was: /usr/lib/network/network: eval: line 153: syntax error: unexpected end of file. I just switched to another solution. Commented Apr 10, 2018 at 6:36
  • There are situations when you really genuinely want a here document, but if you are simply looking for how to put a newline in a static string, probably read stackoverflow.com/questions/3005963/… instead. Commented Mar 28, 2022 at 5:03

16 Answers 16

743

You can avoid a useless use of cat and handle mismatched quotes better with this:

$ read -r -d '' VAR <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF

If you don't quote the variable when you echo it, newlines are lost. Quoting it preserves them:

$ echo "$VAR"
abc'asdf"
$(dont-execute-this)
foo"bar"''

If you want to use indentation for readability in the source code, use a dash after the less-thans. The indentation must be done using only tabs (no spaces).

$ read -r -d '' VAR <<-'EOF'
    abc'asdf"
    $(dont-execute-this)
    foo"bar"''
    EOF
$ echo "$VAR"
abc'asdf"
$(dont-execute-this)
foo"bar"''

If, instead, you want to preserve the tabs in the contents of the resulting variable, you need to remove tab from IFS. The terminal marker for the here doc (EOF) must not be indented.

$ IFS='' read -r -d '' VAR <<'EOF'
    abc'asdf"
    $(dont-execute-this)
    foo"bar"''
EOF
$ echo "$VAR"
    abc'asdf"
    $(dont-execute-this)
    foo"bar"''

Tabs can be inserted at the command line by pressing Ctrl-V Tab. If you are using an editor, depending on which one, that may also work or you may have to turn off the feature that automatically converts tabs to spaces.

In read -r -d '', the options are:

  • -r to accept backslashes literally instead of using them to escape any characters
  • -d sets the first character of the following argument as the delimiter between records (lines), instead of the default which is newline. In the example, I'm setting the delimiter to a null string, so that the whole string passed to read will be read as one record.
Sign up to request clarification or add additional context in comments.

44 Comments

I think it's worth mentioning that if you have set -o errexit (a.k.a set -e) in your script and you use this then it will terminate your script because read returns a non-zero return code when it reaches EOF.
@MarkByers: That's one of the reasons I never use set -e and always recommend against its use. It's better to use proper error handling instead. trap is your friend. Other friends: else and || among others.
Is the avoidance of cat really worth it in such a case? Assigning a heredoc to a variable with cat is a well known idiom. Somehow using read obfuscates things for little benefits imho.
@ulidtko That's because you don't have a space between d and the empty string; bash collapses -rd'' to simply -rd before read ever sees its arguments, so VAR is treated as the argument to -d.
In this format, read will return with a non-zero exit code. This makes this method less than ideal in a script with error checking enabled (eg set -e).
|
428

Use $() to assign the output of cat to your variable like this:

VAR=$(
cat <<'END_HEREDOC'
abc'asdf"
$(dont-execute-this)
foo"bar"''
END_HEREDOC
)

# this will echo variable with new lines intact
echo "$VAR"
# this will echo variable without new lines (changed to space character)
echo $VAR

Making sure to delimit starting END_HEREDOC with single-quotes. This will prevent the content of the heredoc from being expanded, so dont-execute-this will not be executed.

Note that the ending heredoc delimiter, END_HEREDOC, must be alone on the line (hence the ending parenthesis ) is on the next line).

Thanks to @ephemient for the answer.

18 Comments

+1. This is the most readable solution, at least for my eyes. It leaves the name of the variable at the far left of the page, instead of embedding it in the read command.
PSA: remember that the variable must be quoted to preserve newlines. echo "$VAR" instead of echo $VAR.
This is nice with ash and OpenWRT where read doesn't support -d.
For reasons I cannot fathom, this fails with an "unexpected EOF" error if you have an unpaired backtick in the heredoc.
@HubertGrzeskowiak it prevents variable expansion.
|
106

This is a variation of Dennis' method, and it looks more elegant in the scripts.

Function definition (Bash version):

define(){ IFS=$'\n' read -r -d '' ${1} || true; }

Usage:

define VAR <<'EOF'
abc 'asdf " \$ $ \n $'\n'
$(dont-exec ute-this)
foo " bar " ' '

`bad bad ``
EOF

echo "$VAR"

Updated ksh 'read loop' version (also works on Bash):

function define { typeset a o=; while IFS= read -r a; do o+="$a"$'\n'; done; o=${o%?}; eval "$1=\$o"; }

Tests (also includes versions that does not remove the last newline): https://gist.github.com/ootada/e80b6d7fb86acdcda75d77eb7ade364c

14 Comments

This seems to work only superficially. The define function will return a status of 1, and I'm not quite sure what needs to be corrected.
This is also superior to the accepted answer, because it can be modified to support POSIX sh in addition to bash (a read loop in the function, to avoid the -d '' bashism necessary to preserve newlines).
This solution works with set -e set, whereas the selected answer does not. It seems to be because of http://unix.stackexchange.com/a/265151/20650
@fny p.s. return status has long been fixed
ShellCheck SC2141 says it should be define(){ IFS=$'\n' ... (added $)
|
59
VAR=<<END
abc
END

doesn't work, because you are redirecting standard input to something that doesn't care about it, namely the assignment

export A=`cat <<END
sdfsdf
sdfsdf
sdfsfds
END
` ; echo $A

works, but there's a back-tic in there that may stop you from using this. Also, you should really avoid using backticks, it's better to use the command substitution notation $(..).

export A=$(cat <<END
sdfsdf
sdfsdf
sdfsfds
END
) ; echo $A

7 Comments

@l0st3d: So close... Use $(cat <<'END' instead. @Neil: The very last newline will not be part of the variable, but the rest will be preserved.
It doesn't seem like any newlines are preserved. Running the above example I see: "sdfsdf sdfsdf sdfsfds"... ah! But writing echo "$A" (i.e. putting $A in double quotes) and you do see the newlines!
@Darren: aha! I had noticed the newlines issue, and using the quotes around the output variable does fix the issue. thx!
@l0st3d I would just leave out mention of backticks.
Interestingly, due to the quirk of the first example, in a pinch you can use it for makeshift comment blocks like this: REM=<< 'REM' ... comment block goes here ... REM. Or more compactly, : << 'REM' .... Where "REM" could be something like "NOTES" or "SCRATCHPAD", etc.
|
51

There is still no solution that preserves newlines.

This is not true; you're probably just being misled by the behaviour of echo:

echo $VAR # strips newlines

echo "$VAR" # preserves newlines

1 Comment

Really this is the behavior of how quoting a variable works. Without quotes, it will insert them as different parameters, space deliminated, while with quotes the entire variable contents will be treated as one argument
25

Branching off Neil's answer, you often don't need a variable at all. You can use a function in much the same way as a variable, and it's much easier to read than the inline or read-based solutions.

$ complex_message() {
  cat <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF
}

$ echo "This is a $(complex_message)"
This is a abc'asdf"
$(dont-execute-this)
foo"bar"''

2 Comments

This solution is truly awesome. By far the most elegant IMHO.
Lots of options here.
16

An array is a variable, so in that case mapfile will work

mapfile y <<'z'
abc'asdf"
$(dont-execute-this)
foo"bar"''
z

Then you can print like this

printf %s "${y[@]}"

3 Comments

work only for bash 4+ as I can judge
Use cases here for certain.
13

Erman and Zombo are close, but mapfile doesn't just read arrays...

Consider this:

#!/bin/bash
mapfile -d '' EXAMPLE << 'EOF'
Hello
こんにちは
今晩は
小夜なら
EOF
echo -n "$EXAMPLE"

Yielding:

Hello
こんにちは
今晩は
小夜なら

$'' is the delimiter given to mapfile, it will never occur, it means "not delimited".

So there isn't any need for a useless use of cat, and there isn't any need to incur the penalty of recombining arrays.

Furthermore, you get this benefit:

$ echo $EXAMPLE
Hello こんにちは 今晩は 小夜なら

Which you do not receive with Zombo's method:

mapfile y <<'z'
abc'asdf"
$(dont-execute-this)
foo"bar"''
z
echo $y
abc'asdf"

Bonus

If you run it through head -c -1 you can also get rid of that last newline in a way that won't be non-performant:

unset EXAMPLE
mapfile -d '' EXAMPLE < <(head -c -1 << EOF
Hello
こんにちは
今晩は
小夜なら
EOF
)
printf "%q" "$EXAMPLE"
$'Hello\nこんにちは\n今晩は\n小夜なら'

4 Comments

mapfile -d requires bash 4.4 or higher.
Oh, true. If the shell is older than bash 4.4 (Sat, 17 Sep 2016) this won't work, indeed.
Apple only ships bash 3.2 from 2006 :)
Good to know for those who use Apple computers. ;-)
6

In many cases, this page has over-complex answers:

GNUV3="\
    MyProg Copyright (C) $(date +%Y) NVRM 
    This program comes with ABSOLUTELY NO WARRANTY; for details type show w.
    This is free software, and you are welcome to redistribute it
    under certain conditions; type show c for details.
"

echo "$GNUV3"

2 Comments

Cool, except when there are many double quotes within the outer ones and you want shell characters be active at the same time (i. e. $(date ...)) this gets hairy quick.
Sure^. We would want to use intermediary variables to do that. In my opinion, this is also where we should consider other languages than bash itself, as we would start here to mix logic and output template, and we should avoid that anyway. en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller , also true for CLI applications.
4

Assign a heredoc value to a variable

VAR="$(cat <<'VAREOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
VAREOF
)"

Used as an argument of a command

echo "$(cat <<'SQLEOF'
xxx''xxx'xxx'xx  123123    123123
abc'asdf"
$(dont-execute-this)
foo"bar"''
SQLEOF
)"

2 Comments

When I tried the first method, there seems to be no line terminators between the lines. Must be some kind of configuration on my linux machine?
This probably means when you were echoing your variable, you didn't put quotes around it... Try it like so: echo "$VAR"
2

Thanks to dimo414's answer, this shows how his great solution works, and shows that you can have quotes and variables in the text easily as well:

Example execution

./test.sh

Output:

The text from the example function is:
  Welcome dev: Would you "like" to know how many 'files' there are in /tmp?

  There are "      38" files in /tmp, according to the "wc" command

test.sh

#!/bin/bash

function text1()
{
  COUNT=$(\ls /tmp | wc -l)
cat <<EOF

  $1 Would you "like" to know how many 'files' there are in /tmp?

  There are "$COUNT" files in /tmp, according to the "wc" command

EOF
}

function main()
{
  OUT=$(text1 "Welcome dev:")
  echo "The text from the example function is: $OUT"
}

main

1 Comment

It would be interesting to see an unmatched quote in the text to see how it handles that. Maybe ` Don't freak out, there are "$COUNT" files` so the apostrophe/single quote can make things interesting.
1

Others have already posted the approach with cat, but I'll keep it succinct.

somecmdstr=$(cat <<'EOF'
# some bash logic
# . . .
echo "Hi there what's up."
echo "Nothing. What's up with you?"
EOF
)

Then you can echo it or run it. A few different approaches depending on what you want.

$ echo $somecmdstr
# some bash logic # . . . echo "Hi there what's up." echo "Nothing. What's up with you?"

$ bash -c "$somecmdstr"
Hi there what's up.
Nothing. What's up with you?

$ bash -c "echo $somecmdstr"

Hi there what's up.
Nothing. What's up with you?

Comments

1

I found myself having to read a string with NULL in it, so here is a solution that will read anything you throw at it. Although if you actually are dealing with NULL, you will need to deal with that at the hexadecimal level.

File read.dd.sh

read.dd() {
     buf= 
     while read; do
        buf+=$REPLY
     done < <( dd bs=1 2>/dev/null | xxd -p )
     
     printf -v REPLY '%b' $( sed 's/../ \\\x&/g' <<< $buf )
}

Proof:

. read.dd.sh
read.dd < read.dd.sh
echo -n "$REPLY" > read.dd.sh.copy
diff read.dd.sh read.dd.sh.copy || echo "File are different"

HEREDOC example (with ^J, ^M, ^I):

read.dd <<'HEREDOC'
>       (TAB)
>       (SPACES)
(^J)^M(^M)
> DONE
>
> HEREDOC

And:

declare -p REPLY

Output:

declare -- REPLY="  (TAB)
      (SPACES)
(^M)
DONE

"

And:

declare -p REPLY | xxd

Output:

0000000: 6465 636c 6172 6520 2d2d 2052 4550 4c59  declare -- REPLY
0000010: 3d22 0928 5441 4229 0a20 2020 2020 2028  =".(TAB).      (
0000020: 5350 4143 4553 290a 285e 4a29 0d28 5e4d  SPACES).(^J).(^M
0000030: 290a 444f 4e45 0a0a 220a                 ).DONE

Comments

0

This will work in any POSIX shell, avoids the issue of the non-zero exit status of read, and looks cleaner than an inline cat.

def() {
    eval "$1=\$(cat)"
}

def variable << '>>'
abc'asdf"
$(dont-execute-this)
foo"bar"''
>>

Caveat: This will strip newlines at the end of the here-doc. Newlines that are not at the end will be preserved.

2 Comments

def 'shutdown -h now;' <&-?
@TobySpeight The variable should be a variable name (which can't contain spaces or semicolons etc.), otherwise it's a misuse of the def function and this doesn't seem like the type of error that is likely to be made accidentally.
-2

Here's a way to do it that is (in my humble opinion) quite elegant and avoids a UUOC:

  VAR=$(sed -e 's/[ ]*\| //g' -e '1d;$d' <<'--------------------'
      |
      | <!DOCTYPE html>
      | <html>
      |   <head>
      |     <script src='script.js'></script>
      |   </head>
      |   <body>
      |     <span id='hello-world'></span>
      |   </body>
      | </html>
      |
--------------------
    )

The '|' characters define the margin, and only the whitespace to the right of the margin is respected in the printed string. The '1d;$d' strips the first and last line, which are just added as a top and bottom margin around the content. Everything can be indented to whatever level you like, except the HEREDOC delimiter, which in this case is just a bunch of hyphens.

echo "$VAR"

prints

<!DOCTYPE html>
<html>
  <head>
    <script src='script.js'></script>
  </head>
  <body>
    <span id='hello-world'></span>
  </body>
</html>

2 Comments

Um, dude, good attempt but clearly compiling a regular expression and running a global replace is far more resource-hungry of an operation than a UUOC. The mapfile approach is best per appearance and performance, this one is way out there (why the margin??)
While @FredrickBrennan is right, I think that this answer does not deserve three downvotes. Therefore, upvoted. (But indeed keep in mind that this solution is very costly)
-13
$TEST="ok"
read MYTEXT <<EOT
this Bash trick
should preserve
newlines $TEST
long live Perl
EOT
echo -e $MYTEXT

2 Comments

Even with the various errors fixed, this only places the first line of the here document in MYTEXT
Easy to fix, the idea is ok! I used read -d $'\000' MYTEXT <<EOT and worked well...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.