Bash command executed in Python

Question

Consider the following command in bash:

multi-bleu.perl reference < mt-output

I have two questions:

What exactly happens here? My understanding is that reference and mt-output are not arguments, but they are somehow written to stdin. So if they are files, will their contents be written to stdin?
How can I run this in a Python script, and get the result of the Perl script?

My attempt below doesn't work. Included in a file test.py:

from subprocess import Popen, PIPE
import settings

command = settings.BLEU + " " + settings.CORPUS + " < " + settings.HYPOTHESIS
print command
pipe = Popen(command, stdin=PIPE, stdout=PIPE)
output = pipe.communicate()[0]
print output

I run the command printed in the same folder as I run test.py. The former works, the latter doesn't. The error traceback is given below:

Traceback (most recent call last):
  File "/Users/[...]/tests.py", line 6, in <module>
    pipe = Popen([command], stdin=PIPE, stdout=PIPE)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

glglgl · Accepted Answer · 2014-06-24 13:28:36Z

reference is indeed a normal command line argument; its semantics is up to the called program. It might be a file name (input or output), a command or whatever.

Now, there are two ways for calling Popen():

with shell=False (the default):

with open('mt-output', 'r') as infile:
    pipe = Popen(['multi-bleu.perl', 'reference'], stdin=infile, stdout=PIPE)

resp.

with open(settings.HYPOTHESIS, 'r') as infile:
    pipe = Popen([settings.BLEU, settings.CORPUS], stdin=infile, stdout=PIPE)

Here you open the file to be read on your own and pass the opened file for making it the process' standard input.

The file given is called directly.

with shell=True:
```
command = settings.BLEU + " " + settings.CORPUS + " < " + settings.HYPOTHESIS
pipe = Popen(command, stdout=PIPE)
```
Here you give all you do to the shel which lies between your program and the program to be called. Thus, you don't give an argument list, but one command line.

The stdin redirection is performed by the shell as well.

The 2nd solution might seem simpler, but it can be very dangerous depending where the data in settings come from and how they look like.

schaueho · Accepted Answer · 2014-06-24 12:08:58Z

What is happening with your bash / perl example will depend on what the perl script is doing. See the following for an explanation of what is happening on the bash side:

[sugarcube->temp]echo "Foo" > foo
[sugarcube->temp]echo "Bar" > bar
[sugarcube->temp]cat foo < bar
Foo
[sugarcube->temp]cat foo
Foo
[sugarcube->temp]cat bar
Bar
[sugarcube->temp]cat foo - < bar
Foo
Bar

Here I created two files, foo and bar. < does a redirection of file contents to stdin. However, in the first cat example, cat doesn't pick up the (the contents of the redirected file) bar. cat requires an explicit parameter - that tells it to read input from stdin.

So, in your example, reference is just an argument to the perl script (which might open it simply as a file). The content of mt-output is redirected to stdin, so if you Perl script reads from stdin, its going to process it.

If you want to achieve something similar in Python, your Python script would just have to open any file it receives as its first argument (you might want to consider argparse to do regular argument parsing to command line programs, although it might be overkill for this particular problem). For processing data from stdin, just read from sys.stdin.

Collectives™ on Stack Overflow

Bash command executed in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related