Get JSON value from BASH CURL Response with Python

Question

I am having a problem parsing a JSON response to 2 variables in BASH. I dont have access to install jq or jsawk or anything cool that makes life easier. I have python, thats all.

This is what I'm working with: I have a curl call that gets a JSON response. The response is stored in a variable called api_response.

API_RESPONSE=$(curl --silent -v -H "Content-Type: application/json" -H "MY-Token: $Token" -XPOST -d "$INPUTS" ${MY_BASE}$MY_PROCESS${PROCESS})

this variable essentially is the value of the response from the api

[{"name":"test-name1", "value" : "test-value1"},{"name" : "test-name2","value" : "test-value2"}]

In the past, I had only need to get a single value from the response and was able to do that using the following

API_RESPONSE=$(curl --silent -v -H "Content-Type: application/json" -H "MY-Token: $Token" -XPOST -d "$INPUTS" ${MY_BASE}$MY_PROCESS${PROCESS} | python -c "import sys, json; print json.load(sys.stdin)[1]['value'])

[outputs]

test-value2

I was trying to extract to two JSON values from the single variable API_RESPONSE but I get an error doing it this way.

API_RESPONSE=$(curl --silent -v -H "Content-Type: application/json" -H "MY-Token: $Token" -XPOST -d "$INPUTS" ${MY_BASE}$MY_PROCESS${PROCESS})

myvar1=$($API_RESPONSE | python -c "import sys, json; print json.load(sys.stdin)[0]['value']")
myvar2=$($API_RESPONSE | python -c "import sys, json; print json.load(sys.stdin)[1]['value']")

I get the following error:

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib64/python2.6/json/__init__.py", line 267, in load 
parse_constant=parse_constant, **kw)
File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads return _default_decoder.decode(s)
File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.6/json/decoder.py", line 338, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

The variable api_response is the same data that was working before. Why would it work with a curl call and not from variable?

Im not in a position to argue about the age of the technology. I just need insight on why python works with curl in-line, and not with a variable. — Autonomic
– Autonomic, Commented Jan 4, 2017 at 13:26

Autonomic · Accepted Answer · 2017-01-05 13:36:27Z

I figured it out with help from a friend.

When going from bash variable to python sys.stdin, a bash variable needs to be redirected by SOME action to the python sys.stdin.

With the curl execution piping output to the python sys.stdin it would work because the output was being redirected to python. But once I had the entire response stored in the variable within bash, the next step was to echo the output and redirect to python.

[json response]

[{"name":"test-name1", "value" : "test-value1"},{"name" : "test-name2","value" : "test-value2"}]

[code block]

API_RESPONSE=$(curl --silent -v -H "Content-Type: application/json" -H "MY-Token: $Token" -XPOST -d "$INPUTS" ${MY_BASE}$MY_PROCESS${PROCESS})

myvar1=$( echo $API_RESPONSE | python -c "import sys, json; print json.load(sys.stdin)[0]['value']")
myvar2=$( echo $API_RESPONSE | python -c "import sys, json; print json.load(sys.stdin)[1]['value']")
echo $myvar1
echo $myvar2

[outputs]

test-value1
test-value2

Collectives™ on Stack Overflow

Get JSON value from BASH CURL Response with Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related