How to pipe output from a linux command to a python script?

Question

We have a python script that accepts the following arguments as example:

mypie.py --size=20 --pielist='{"apple":[5],"orange":[7]}' --quantity=10

We tried in our test.sh bash shell script:

test.sh 20 5 7 10

test.sh

export SIZE=$1
export APPLE=$2
export ORANGE=$3
export QUANTITY=$4

echo "--size=$SIZE" --pipelist='{\"apple\":[$2],\"orange\":[$3]}\"' --quantity=$4" | tee output.txt
cat output.txt | sudo python mypie.py

We got error ERROR: size is not specified.

but when we cat output.txt, we can see that it is there with size value.

--size=20 --pielist='{"apple":[5],"orange":[7]}' --quantity=10

What are we doing wrong ? Thanks

What do you mean "doesn't work" what's the output ? How do you read the args in mypie.py ? — Nir Alfasi
– Nir Alfasi, Commented Aug 7, 2014 at 6:12
@alfasin: the problem is that the OP wants to pass the contents of output.txt as arguments, not as standard input. — tripleee
– tripleee, Commented Aug 7, 2014 at 6:54

Yogeesh Seralathan · Accepted Answer · 2014-08-07 05:55:45Z

3

This would do -

cat output.txt | sudo xargs python mypie.py

answered Aug 7, 2014 at 5:55

Yogeesh Seralathan

1,4244 gold badges16 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

chz Over a year ago

Sorry, user abc is not allowed to execute '/usr/bin/xargs

tripleee Over a year ago

The cat is useless, though. sudo xargs python mypie.py <output.txt

tripleee Over a year ago

If you don't have sudo privileges to run xargs, maybe try xargs sudo python mypie.py <output.txt instead.

Yogeesh Seralathan Over a year ago

Exactly what @tripleee said would work. Check with this and get back.

parade · Accepted Answer · 2014-08-07 06:50:36Z

3

Using the pipe redirects the content to stdin of the program - not the command line.
You could consider building up an args variable - then passing that to both the output.txt, and your python script.
For example:

export SIZE=$1
export APPLE=$2
export ORANGE=$3
export QUANTITY=$4

export ARGS="--size=$SIZE" --pipelist='{\"apple\":[$2],\"orange\":[$3]}\"' --quantity=$4"

echo $ARGS | tee output.txt
sudo python mypie.py $ARGS

edited Aug 7, 2014 at 6:50

answered Aug 7, 2014 at 6:23

parade

1063 bronze badges

4 Comments

chz Over a year ago

Hi thanks for responding - we got error ERROR: 'str' object has no attribute 'has_key'

parade Over a year ago

Potentially that's because the pipelist wasn't correctly escaped .. i.e you need to escape the script & the command line - insert additional \'s. export ARGS="--size=$SIZE" --pipelist='{\\"apple\\":[$2],\\"orange\\":[$3]}\\"' --quantity=$4"

tripleee Over a year ago

Is it really, really, really necessary to have the parameters in a variable, though? Simply hard-coding the arguments makes the code simpler, more readable, and more maintainable. See also mywiki.wooledge.org/BashFAQ/050

Yogeesh Seralathan Over a year ago

tee thats new for me. That simplies a lot of my prev code's. Thanks for tee.

Dunes · Accepted Answer · 2014-08-07 06:12:00Z

1

You can use command substitution to directly place the output of command as the arguments to your python program.

python mypie.py `cat output.txt`

or

python mypie.py $(cat output.txt)

answered Aug 7, 2014 at 6:12

Dunes

42.1k7 gold badges86 silver badges107 bronze badges

2 Comments

chz Over a year ago

Hi Dunes thanks for responding…It says ERROR: 'str' object has no attribute 'has_key'. Is our escaping single quotes correct - output.txt shows correct argument list

Dunes Over a year ago

Python does not automatically evaluate expression on the command line, it just passes the raw string. To turn the string into a dict you need to use eval, which has security implications, or the more secure loads in the json module.

tripleee · Accepted Answer · 2014-08-07 07:07:57Z

1

The simple and obvious solution is to change test.sh so that it passes the parameters correctly.

#!/bin/sh
set -x  # if you want to see the script's parameters
sudo python mypie.py --size="$1" \
            --pipelist="{'apple':[$2],'orange':[$3]}" --quantity="$4"

Passing the arguments as standard input (which is what a pipe does) is simply not how it is usually done, nor a particularly suitable model for this particular type of interaction.

edited Aug 7, 2014 at 7:07

answered Aug 7, 2014 at 7:01

tripleee

192k37 gold badges318 silver badges367 bronze badges

1 Comment

tripleee Over a year ago

I switched to single quotes around the dictionary keys in pipelist for legibility reasons. If they absolutely have to be double quotes, you need to backslash-escape them.

Collectives™ on Stack Overflow

How to pipe output from a linux command to a python script?

4 Answers 4

4 Comments

4 Comments

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

4 Comments

4 Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related