subprocess.check_output() module object has out attribute 'check_output'

Question

I am playing with the subprocess library. I can perform

v = subprocess.check_output("ls")

and it works fine. However, when I SSH into a machine and I want to perform a hadoop command,

v = subprocess.check_output("hadoop fs -ls /path/to/file* | grep -oE '/(.*)'")

I get

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'check_output'

Not quite sure why. I think it might be my lack of understanding of subprocess. How can i fix this error?

what version of python on the remote machine?

Padraic Cunningham
– Padraic Cunningham

2014-11-12 18:37:03 +00:00
Commented Nov 12, 2014 at 18:37 — Padraic Cunningham
– Padraic Cunningham, Commented Nov 12, 2014 at 18:37
@PadraicCunningham 2.6.7

Liondancer
– Liondancer

2014-11-12 18:37:49 +00:00
Commented Nov 12, 2014 at 18:37 — Liondancer
– Liondancer, Commented Nov 12, 2014 at 18:37

Padraic Cunningham · Accepted Answer · 2014-11-12 18:50:04Z

21

check_output was introduced in python 2.7 so won't work for < 2.7.

You can use Popen with communicate to get the output.

from subprocess import PIPE,Popen
proc = Popen(['ls', '-l'], stdout=PIPE)
print(proc.communicate()[0].split())

This part of the docs may be useful

edited Nov 12, 2014 at 18:50

answered Nov 12, 2014 at 18:38

Padraic Cunningham

181k30 gold badges264 silver badges327 bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

Liondancer Over a year ago

Would you happen to know an alternative? I simply want to get the output into a list

Padraic Cunningham Over a year ago

I will add something in a couple of minutes

Liondancer Over a year ago

I read around about Popen and that deadlocks occur? This is something I am worried about. How could I prevent this from occuring?

Padraic Cunningham Over a year ago

do you mean using multiple processes?

Bakuriu Over a year ago

@Liondancer If you look at the sources of check_output you'll see that it does exactly what described in the answer. It has a bunch of other code to set up the arguments and handle exceptions, but what it does is simply create a Popen and call communicate. The note you are referring to is only valid when the subprocess produces a lot of data. In such cases you have to read "lazily" from the process stdout to avoid the buffering.

|

v.babak · Accepted Answer · 2018-09-28 06:42:02Z

3

Here is my version of subprocess.check_output function for Python 2.6 and it supports pipe lines along with habitual command interface.

import subprocess

def check_output(cmd):
    """ https://docs.python.org/2/library/subprocess.html#subprocess.Popen
        Implementation subprocess.check_output() for Python 2.6
     """
    process_list = []
    cmd_list = cmd.strip().split("|")
    for i, sub_cmd in enumerate(cmd_list):
        STDIN = None
        if i > 0:
            STDIN = process_list[i - 1].stdout
        process_list.append(subprocess.Popen(sub_cmd, stdin=STDIN, stdout=subprocess.PIPE, shell=True))
    if len(process_list) == 0:
        return ''
    output = process_list[i].communicate()[0]
    return output

Test call:

print check_output('ls -la /var | grep log')

Output:

drwxr-xr-x 1 root root 4096 Dec 11 13:22 log

edited Sep 28, 2018 at 6:42

answered Jan 9, 2018 at 16:48

v.babak

93612 silver badges14 bronze badges

1 Comment

v.babak Over a year ago

Added "shell=True" and removed unnecessary lines. That will fix running "curl" commands.

MVanOrder · Accepted Answer · 2018-06-15 21:53:11Z

2

padraic-cunningham, while your answer got me part of the way there, in my case the command is an SSH command. Since communicate kills my process prematurely(as soon as it reads an EOF character), I've found the following works better:

import subprocess

def check_output(command_list):
    proc = subprocess.Popen(command_list, stdout=subprocess.PIPE)
    proc.wait()
    return proc.stdout.read()

answered Jun 15, 2018 at 21:53

MVanOrder

1,4431 gold badge11 silver badges21 bronze badges

Collectives™ on Stack Overflow

subprocess.check_output() module object has out attribute 'check_output'

3 Answers 3

13 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

13 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related