Bash : Curl grep result as string variable

Question

I have a bash script as below:

curl -s "$url" | grep "https://cdn" | tail -n 1 | awk -F[\",] '{print $2}'

which is working fine, when i run run it, i able to get the cdn url as:

https://cdn.some-domain.com/some-result/

when i put it as variable :

myvariable=$(curl -s "$url" | grep "https://cdn" | tail -n 1 | awk -F[\",] '{print $2}')

and i echo it like this:

echo "CDN URL:  '$myvariable'"

i get blank result. CDN URL:

any idea what could be wrong? thanks

A common problem is that some HTTP APIs output DOS line endings, though I don't see how you could get exactly the output you claim because of that. (But then your diagnostics don't seem particularly exact.) — tripleee
– tripleee, Commented Mar 23, 2021 at 7:23

tripleee · Accepted Answer · 2021-03-23 07:29:12Z

2

If your curl command produces a trailing DOS carriage return, that will botch the output, though not exactly like you describe. Still, maybe try this.

myvariable=$(curl -s "$url" | awk -F[\",] '/https:\/\/cdn/{ sub(/\r/, ""); url=$2} END { print url }')

Notice also how I refactored the grep and the tail (and now also tr -d '\r') into the Awk command. Tangentially, see useless use of grep.

answered Mar 23, 2021 at 7:29

tripleee

192k37 gold badges318 silver badges367 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Teddybugs Over a year ago

you are right, the ^M character cause it become invalid url for CURL, thanks.

PromethiumL · Accepted Answer · 2021-03-23 07:27:42Z

0

The result could be blank if there's only one item after awk's split. You might try grep -o to only return the matched string:

myvariable=$(curl -s "$url" | grep -oP 'https://cdn.*?[",].*' | tail -n 1 | awk -F[\",] '{print $2}')
echo "$myvariable"

edited Mar 23, 2021 at 7:27

answered Mar 23, 2021 at 7:11

PromethiumL

133 bronze badges

1 Comment

tripleee Over a year ago

Not my downvote, but your regex seems frightfully wrong for what you are suggesting. Also, switching from proper modern $(...) command-substitution syntax to obsolescent backticks seems like a move in the wrong direction. Finally, you should quote your shell variable.

Collectives™ on Stack Overflow

Bash : Curl grep result as string variable

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related