Run consecutive Shell commands in Python-Subprocess

Question

These are some dependant commands i am trying to run. My expectation was it will change current folder to abc & list files.

Also after setting z=88, it will print z.

import subprocess
cmd_list = []
cmd_list.append("cd ./abc")
cmd_list.append("ls")
cmd_list.append("export z=88")
cmd_list.append("echo $z")

my_env = os.environ.copy()
for cmd in cmd_list:
    sp = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=my_env, shell=True,text=True)

But unable to get any output for ls and echo $z

The commands are being run in a child process. The changes they make don't have any effect on the parent Python process. — Barmar
– Barmar, Commented Apr 12, 2020 at 20:13
Exported variables are only inherited by children of the shell process, they don't go back to the Python process, and won't be inherited by another child of the Python process. — Barmar
– Barmar, Commented Apr 12, 2020 at 20:14

Barmar · Accepted Answer · 2020-04-12 20:23:41Z

2

You can't do this with multiple calls to subprocess.Popen(). Each call creates a new child process, and changes they make to their environment do not propagate back to the Python process.

You can do it by concatenating all the commands into a single command line to be run by bash -c.

cmd = 'cd abc; ls; export z=88; echo $z'
subprocess.Popen(['bash', '-c', cmd], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=my_env,text=True)

There's also no point in exporting z, since you're not running any child processes of the shell that use the environment variable. Just assign an ordinary shell variable with z=88.

edited Apr 12, 2020 at 20:23

answered Apr 12, 2020 at 20:18

Barmar

789k57 gold badges554 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Gabio Over a year ago

Are you sure that stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE and capture_output work together?

Barmar Over a year ago

Not really, I just copied that from the question. I removed it. I also removed shell=True, which it not useful when the first argument is a list.

Bond Over a year ago

Yeah, i mistakenly added capture_output. @Barmar Is there any way i can split that command ? Because for example, let's say if first command ran successfully and second didn't then i should revert first commands results.

Barmar Over a year ago

You can separate commands with && instead of ; so the next command only runs if the previous one was successful.

Collectives™ on Stack Overflow

Run consecutive Shell commands in Python-Subprocess

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related