Printing output from grep in bash script yields broken string

Question

I have the following simple bash script that stores the output of a command in a variable and prints it:

size=$(curl -sI "http://speedtest.reliableservers.com/1GBtest.bin" | grep -i "length")

echo "--> ${size} <--"

When running the command in the terminal, I get the following output:

Content-Length: 1073471824

But when I run this bash script that invokes the command, I get the following output:

 <--Content-Length: 1073741824

What is going on?

I run it as ./wtf.sh in the Ubuntu terminal. Both sh wtf.sh and bash wtf.sh result in the same thing. — Overv
– Overv, Commented Dec 7, 2015 at 16:00
@Kenney Both result in "ASCII text, with CRLF line terminators" for me. Vim shows that 2.txt has just a single line, however. — Overv
– Overv, Commented Dec 7, 2015 at 16:04

Kenney · Accepted Answer · 2015-12-07 16:36:59Z

5

The problem is that the HTTP response header ends with CRLF or \r\n. The $(..) does strip the \n but not the \r, so that the output will be

--> 1073741824\r <--

where the \r carriage return sets the cursor to the start of the line, overwriting --> with <--.

You can strip the \r with sed:

size=$(curl -sI "http://speedtest.reliableservers.com/1GBtest.bin" | grep -i "length" \
     | sed -e 's/\r//' )

echo "--> ${size} <--"

answered Dec 7, 2015 at 16:07

Kenney

9,11317 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

That solves the problem, thanks. Is there no cleaner way to get the result than stripping the \r like this?

Cleaner how? There are certainly other ways to strip it; tr -d '\015' comes immediately to mind.

@tripleee I was thinking of some flag that forces cURL to output unix line endings.

I don't think there is one, but grep -io '^content-length: [0-9]*' would not capture and thus not print the carriage return in the first place.

@Overv Depending on what you actually need, the --write-out option may be useful.