How to use linux command in subprocess python

Question

I have linux command like below:

 find /data/*/hr/ -printf "%f: %p: %u: %g %m (%M) \n"

How do i use in python subprocess check_output

I have tried like below but not working

 file_name = "/data/*/%s/" % (filename)
 get_perm = check_output(["find", file_name, "-printf", '\"%f: %p: %u: %g %m (%M) \n\"'])

Error I am getting:

find: ‘/data/*/hr/’: No such file or directory
Traceback (most recent call last):
  File "handover.py", line 90, in <module>
    get_perm = check_output(["find", file_name, "-printf", '\"%f: %p: %u: %g %m (%M) \n\"'])
  File "/usr/lib64/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['find', '/data/*/hr/', '-printf', '"%f: %p: %u: %g %m (%M) \n"']' returned non-zero exit status 1

Do you get any output? Do you get an error (i.e. a CalledProcessError?). If it dumped because it didn't like the -printf"%f: %p:... argument, it should have raised something — Nick T
– Nick T, Commented Sep 10, 2017 at 23:40
Python is also translating \n in the string into a literal newline, which might be strange when find gets it. You can also avoid escaping " chars by using ' chars to wrap the string literal in. — Nick T
– Nick T, Commented Sep 10, 2017 at 23:42
Possible duplicate of Running shell command from Python and capturing the output — dlmeetei
– dlmeetei, Commented Sep 11, 2017 at 2:55

gopinara · Accepted Answer · 2017-09-11 05:35:22Z

2

Finally,

I found below method

cmd = "find /data/*/{}/* -printf \"%f:%p:%u:%g:%m\n\"".format(filename)
info = subprocess.Popen(cmd,stdout=subprocess.PIPE,shell=True)
print info.stdout.read()

This solves my problem

answered Sep 11, 2017 at 5:35

gopinara

518 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Michael Mior · Accepted Answer · 2017-09-11 02:15:24Z

0

Your shell will expand the * in /data/*/hr/ when you call on the command line. Calling the function directly via check_output causes find to look for literally the directory /data/*/hr/. You could use the glob module to expand the path before passing to find:

import glob

file_name = "/data/*/%s/" % (filename)
get_perm = check_output(["find"] + glob.glob(file_name) + ["-printf", '\"%f: %p: %u: %g %m (%M) \n\"'])

glob.glob simply produces an array of path names which match the given expression by expanding any *s and some other special characters.

answered Sep 11, 2017 at 2:15

Michael Mior

28.9k10 gold badges95 silver badges118 bronze badges

4 Comments

gopinara Over a year ago

Hi Michael, It works fine.. But problem is filename "/data/*/fin/" . If fin does not exist. It will list other values. If fin does not exit. I expect to get no such file or directory error

Peter Kirby Over a year ago

@gopinara It will list other files because you have a trailing slash, which the find command interprets as you indicating you want to list all of the files/directories contained within /fin/. Try removing the trailing slash. You can also specify a type to find to limit the results to only files or directories, like, find /data/*/fin -type f. Also, the first argument to find is usually the directory which you want to search within for a file or directory. Maybe you want something like find /data/*/fin -type f -name your_file_name.ext

gopinara Over a year ago

I tried removing slash and adding -type d..same result

Michael Mior Over a year ago

You may just want to check if glob.glob(file_name) produces an empty array and if so, don't run the command at all.

Peter Kirby · Accepted Answer · 2017-09-11 03:57:26Z

0

You are getting this error because the specified file does not exist. If you run the command directly in a shell, you will get the same response.

For example, the following works as expected:

import subprocess
import os

file_name = os.path.join(os.getcwd(), 'test.txt')
with open(file_name, 'w') as f:
    f.write('hello world')

get_perm = subprocess.check_output([
    "find",
     file_name,
     "-printf",
    '"%f: %p: %u: %g %m (%M) \n"'
    ], shell=True)

print(get_perm)
os.remove(file_name)

According to the docs:

If the return code [from subprocess.check_output] was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and any output in the output attribute.

I recommend you wrap your check_output call in a try..except, and catch the CalledProcessError.

Alternatively, if you really don't want to deal with the exception, you could instead execute the command:

x=$(find ~/data/*/hr/ -printf "%f: %p: %u: %g %m (%M) \n" 2>/dev/null || true) && echo $x

This will never return nonzero and will only ever contain output if the file exists.

Edit As Michael pointed out, the '*' is not getting expanded. However, if you set shell=True, it will. Try modifying your command as follows:

 file_name = "/data/*/%s/" % (filename)
 get_perm = check_output(["find", file_name, "-printf", '"%f: %p: %u: %g %m (%M) \n"'], shell=True)

edited Sep 11, 2017 at 3:57

answered Sep 11, 2017 at 2:05

Peter Kirby

1,9942 gold badges16 silver badges29 bronze badges

3 Comments

Michael Mior Over a year ago

While it's true that the file isn't found, the problem is likely that the * in the path should be expanded.

Peter Kirby Over a year ago

@MichaelMior Good catch. If you set shell=True, it will expand that the splat.

Michael Mior Over a year ago

You're right. Adding shell=True is an easy solution :)

Collectives™ on Stack Overflow

How to use linux command in subprocess python

3 Answers 3

Comments

4 Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

4 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related