python subprocess run works with single string but not list of strings

Question

I'm trying to initiate a command-line program with command-line options from my Python script using the run method from the subprocess module.

My command is defined as a list of strings specifying the program and options as follows (where pheno_fp and construction_fp are strings representing file paths in my system and exe is a string representing the file path to the program I'm running):

    step1_cmd = [exe, 
                "--step 1",
                "--p " + pheno_fp,
                "--b 1000",
                "--o " + construction_fp + "dpw_leaveout"]

Not working - When I try the following, the program I want to run is started but the command I've specified is interpreted incorrectly because the program exits with an error saying "specify output file path with --o flag":

    test1 = subprocess.run(step1_cmd)

Working - When I try the following, the program executes correctly, meaning all arguments are interpreted as intended:

    test1 = subprocess.run(" ".join(step1_cmd), shell=True)

If I understood the documentation correctly, the former approach is the recommended usage but I can't understand why it's not working. I'm pretty sure it's formatted the same as the examples in the documentation so I'm a bit stumped. Any ideas?

try to separate each parm with a comma if you want to use it as list — Suryaveer Singh
– Suryaveer Singh, Commented Aug 25, 2020 at 15:41

Zionsof · Accepted Answer · 2020-08-25 15:39:24Z

5

Split each parameter with its value, like so:

step1_cmd = [exe, 
             "--step",
             "1",
             "--p",
             str(pheno_fp),  # if it isn't a string already
             "--b",
             "1000",
             "--o",
             str(construction_fp) + "dpw_leaveout"
]

Because when passing a list of parameters, each part is separated with a space, both options and their values

edited Aug 25, 2020 at 15:39

answered Aug 25, 2020 at 15:38

Zionsof

1,27613 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

larsks Over a year ago

You've got some typos there, and I think all your arguments need to be strings.

Zionsof Over a year ago

Right, forgot about that. Updated the answer

user2314737 · Accepted Answer · 2020-08-25 16:22:34Z

The explanation for this behavior is here:

args is required for all calls and should be a string, or a sequence of program arguments. Providing a sequence of arguments is generally preferred, as it allows the module to take care of any required escaping and quoting of arguments (e.g. to permit spaces in file names).

Example: the sequence

l = ['ls', '-l tmp']

gives an error

subprocess.run(l)                                                                                                                                                      
ls: illegal option --

This is because subprocess (which is a call to Popen) is trying to run ls "-l tmp" .

The correct way to define a sequence of arguments is by separating them so that they can be quoted correctly

subprocess.run(['ls', '-l', 'tmp'])

Collectives™ on Stack Overflow

python subprocess run works with single string but not list of strings

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related