Python subprocess to call Unix commands, a question about how output is stored

Question

I am writing a python script that reads a line/string, calls Unix, uses grep to search a query file for lines that contain the string, and then prints the results.

from subprocess import call

for line in infilelines:

    output = call(["grep", line, "path/to/query/file"])
    print output
    print line`

When I look at my results printed to the screen, I will get a list of matching strings from the query file, but I will also get "1" and "0" integers as output, and line is never printed to the screen. I expect to get the lines from the query file that match my string, followed by the string that I used in my search.

Jonathan Sternberg · Accepted Answer · 2011-03-25 21:49:04Z

6

call returns the process return code.

If using Python 2.7, use check_output.

from subprocess import check_output
output = check_output(["grep", line, "path/to/query/file"])

If using anything before that, use communicate.

import subprocess
process = subprocess.Popen(["grep", line, "path/to/query/file"], stdout=subprocess.PIPE)
output = process.communicate()[0]

This will open a pipe for stdout that you can read with communicate. If you want stderr too, you need to add "stderr=subprocess.PIPE" too.

This will return the full output. If you want to parse it into separate lines, use split.

output.split('\n')

I believe Python takes care of line-ending conversions for you, but since you're using grep I'm going to assume you're on Unix where the line-ending is \n anyway.

http://docs.python.org/library/subprocess.html#subprocess.check_output

edited Mar 25, 2011 at 21:49

answered Mar 25, 2011 at 19:29

Jonathan Sternberg

6,7177 gold badges43 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Fred Foo Over a year ago

+1. The extra batteries in Python 2.7 keep pleasantly surprising me.

brokentypewriter Over a year ago

I used the code that you wrote above. When I print output I get every line from my query file, and not only those lines that match my string. I read the documentation that you pointed to (however, I have to say I don't really understand it, and I had read it before I posted my initial question). I am looking for a more descriptive explanation of communicate.

Jonathan Sternberg Over a year ago

Screwed up my code. You should get 0 lines (output should be None). I'll update with what's necessary. If you're getting every line, then you might not be using grep correctly.

João Pinto · Accepted Answer · 2011-03-25 22:34:32Z

4

The following code works with Python >= 2.5:

from commands import getoutput
output = getoutput('grep %s path/to/query/file' % line)
output_list = output.splitlines()

answered Mar 25, 2011 at 22:34

João Pinto

5,6475 gold badges23 silver badges36 bronze badges

1 Comment

wordsforthewise Over a year ago

Deprecated since Python 2.6. Use subprocess instead. docs.python.org/2/library/commands.html

kurumi · Accepted Answer · 2011-03-26 01:24:30Z

-2

Why would you want to execute a call to external grep when Python itself can do it? This is extra overhead and your code will then be dependent on grep being installed. This is how you do simple grep in Python with "in" operator.

query=open("/path/to/query/file").readlines()
query=[ i.rstrip() for i in query ]
f=open("file")
for line in f:
    if "line" in query:
        print line.rstrip()
f.close()

answered Mar 26, 2011 at 1:24

kurumi

25.7k5 gold badges47 silver badges52 bronze badges

1 Comment

makhdumi Over a year ago

Because python can be far slower than grep when working with large files

Collectives™ on Stack Overflow

Python subprocess to call Unix commands, a question about how output is stored

3 Answers 3

3 Comments

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related