Passing bash variable to python command within a shell script

Question

Needing to extract two variables per line from a JSON file and use each of these 2 variables in separate follow up commands.

My script is so far:

#!/bin/bash

VCTRL_OUT='/tmp/workingfile.json'

old_IFS=$IFS      # save the field separator
IFS=$'\n'     # new field separator, the end of line

for line in $(cat $VCTRL_OUT)
  do
    python -c 'import json,sys;obj=json.load($line);print obj["ip_range"]'
  done

The second to last line is wrong and need to know how to do this.

The following works:

cat /tmp/workingfile.json | head -n +1 | python -c 'import json,sys;obj=json.load(sys.stdin);print obj["ip_range"]';

But not sure how to do the same in the bash script for loop.

Single quotes in bash kill all special characters, even the $ in $line variable, so instead of the variable's value, you are passing a "$line" string. You need to use double quotes around the python -c ... command. — karlosss
– karlosss, Commented Sep 21, 2016 at 18:59
What does the JSON file look like? There are better options than trying to run a separate Python program for every line of the file. (Ignoring the fact that JSON is not line-oriented and cannot generally be processed line-by-line, this is the wrong way to iterate over a file in bash; see Bash FAQ 001.) — chepner
– chepner, Commented Sep 21, 2016 at 19:18
@karlosss - the json.load command that runs now does not work since it is running against the json line. What needs to be modified ? — dross
– dross, Commented Sep 21, 2016 at 20:16
@chepner - Agreed the wrong way to iterate over a file. The contents looks like this: {"ip_range":"1.1.1.1/24","uuid":"b4d8c9344ffb010aec2e93a78335639f","description":"string: Yellow"} {"ip_range":"2.2.2.2/26","uuid":"b4d8c9344ffc0e0ec77893a78335639f","description":"string: Red"} — dross
– dross, Commented Sep 21, 2016 at 20:24
This seems rather convoluted, why the bash layer and not just write a single python script to do it? — proycon
– proycon, Commented Sep 21, 2016 at 20:46

chepner · Accepted Answer · 2016-09-22 11:39:09Z

3

Python wouldn't be my first choice for this kind of one-liner, but you might try

#!/bin/bash

VCTRL_OUT='/tmp/workingfile.json'

parse_json () {
    python -c $'import json,fileinput,operator\nfor line in fileinput.input(): print "%s %s"%operator.itemgetter("ip_range","description")(json.loads(line))' "$1"
}
while IFS= read -r ip_range description; do
   # do your thing with ip_range
done < <(parse_json "$VCTRL_OUT")

One alternative is to replace the Python bit with jq:

while IFS= read -r ip_range description; do
  # ...
done < <( jq -rRc 'fromjson | .ip_range+" "+.description' "$VCTRL_OUT")

Another alternative is to replace the entire bash script with Python, although that may be easier said than done, depending on what the bash script is actually doing.

edited Sep 22, 2016 at 11:39

answered Sep 21, 2016 at 21:02

chepner

538k77 gold badges594 silver badges746 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Passing bash variable to python command within a shell script

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related