Using PIPE in long subprocess call (python) doesn't work

Question

I am trying to do the following command (to download Calibre through a python script):

sudo -v && wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py | sudo python -c "import sys; main=lambda:sys.stderr.write('Download failed\n'); exec(sys.stdin.read()); main()"

Following some of the answers on here on how to PIPE, I have been doing this:

import subprocess
from subprocess import Popen, PIPE
wget = subprocess.Popen(["sudo -v wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py"], stdout=PIPE) 
run = subprocess.Popen(["sudo python -c "import sys; exec(sys.stdin.read()); main()""], stdin=wget.stdout)

I have tried changing many things but NOTHING is working. There are too many errors to put here. Can anyone correct this? Many thanks in advance.

All I get is the first

-1. "NOTHING is working" -- the code in your question doesn't show much effort e.g., it contains SyntaxErrors (quotes usage). You should fix them before posting a question. The Popen() usage in your code is also incorrect: at the very least you should read the description of the first parameter args that specifies the command to run. Is the phrase "a sequence of program arguments" not sufficiently clear? Why do you call Popen(["cmd arg1 arg2"]) instead of Popen(["cmd", "arg1", "arg2"])? — jfs
– jfs, Commented Jun 25, 2014 at 16:51
It may not show effort, but saying "I have tried changing many things" is better than showing you all a HUGE message describing every little error I get on each slight alteration of the code. If you want, I can email you the whole list. — user3775491
– user3775491, Commented Jun 26, 2014 at 14:47

bakkal · Accepted Answer · 2014-06-25 14:17:04Z

When you have parameters you need to break it into a list, e.g.

wget = subprocess.Popen(["wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py"]) 

(**No such file or directory** error, because it looks for that whole string as a command/file)

Needs to become:

subprocess.Popen(['wget', '-nv', '-O-', 'https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py'])
<subprocess.Popen object at 0x10e203950>

You can also use shlex.split() to split your command for you, e.g.

>>> import shlex
>>> shlex.split('wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py')
['wget', '-nv', '-O-', 'https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py']

Ref. https://docs.python.org/2/library/subprocess.html#popen-constructor

You also have "sudo python -c "import sys; exec(sys.stdin.read()); main()"", I don't think it's acceptable syntax because you have quotes inside quotes (the inside ones close out the first opening quote), so try 'sudo python -c "import sys; exec(sys.stdin.read()); main()"' with single quotes instead, that way you don't have to escape the quotes inside!

jfs · Accepted Answer · 2014-06-25 16:52:03Z

To avoid escaping quotes inside the command, you could use triple quotes:

from subprocess import check_call

check_call(r'''# http://calibre-ebook.com/download_linux
  sudo -v &&
  wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py |
  sudo python -c "import sys; main=lambda:sys.stderr.write('Download failed\n'); exec(sys.stdin.read()); main()"
''', shell=True)

shell=True causes the first parameter to be interpreted as a shell command.

In principle, you could simplify the command:

#!/usr/bin/env python3
import sys
from subprocess import check_call
from urllib.request import urlretrieve

# http://calibre-ebook.com/download_linux
path, _ = urlretrieve('https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py')
check_call(['sudo', sys.executable, path])

but it may have subtle security implications e.g., urlretrieve() may skip server ssl certificate validation.

Collectives™ on Stack Overflow

Using PIPE in long subprocess call (python) doesn't work

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related