Error while executing bash for loop from python subprocess

Question

I want to run this command from python mentioned here:

ffmpeg -f concat -safe 0 -i <(for f in ./*.wav; do echo "file '$PWD/$f'"; done) -c copy output.wav

But i can't even run this:

subprocess.run('for i in {1..3}; do echo $i; done'.split(), capture_output=True)

Error:

Traceback (most recent call last):
  File "/media/russich555/hdd/Programming/Freelance/YouDo/21.intercom_record/test.py", line 36, in <module>
    pr = subprocess.run(
         ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 546, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1022, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.11/subprocess.py", line 1899, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'for'

Process finished with exit code 1

Also tried add shell=True:

subprocess.run('for i in {1..3}; do echo $i; done'.split(), capture_output=True, shell=True)

stderr output:

i: 1: Syntax error: Bad for loop variable

Also tried pass /bin/bash, because documentation says that shell=True using /bin/sh

subprocess.run('/bin/bash for i in {1..3}; do echo $i; done'.split(), capture_output=True)

stderr output:

/bin/bash: for: No such file or catalog

Does this answer your question? for loop in `Subprocess.run` results in `Syntax error: "do" unexpected` — Constantin Hong
– Constantin Hong, Commented Dec 4, 2022 at 7:59
When using shell=True do not use "some bash command".split(). — Socowi
– Socowi, Commented Dec 4, 2022 at 8:51
Also, separately, plain split is wrong; if you need to split a command line, use shlex.split() — tripleee
– tripleee, Commented Dec 4, 2022 at 10:32

tripleee · Accepted Answer · 2022-12-04 10:40:04Z

1

There are two errors here, or really, three;

You are trying to use shell features without shell=True
You are trying to use Bash features, but the default shell on non-Windows platforms is POSIX sh; you can fix that with executable='/bin/bash' (obviously, adjust the path if necessary).

More fundamentally, though, you want to avoid using a subprocess when Python can perform the loop natively.

from pathlib import Path
import subprocess

subprocess.run(
    ['ffmpeg', '-f', 'concat', '-safe', '0',
     '-i', '/dev/stdin', '-c', 'copy', 'output.wav'],
    input="".join(f"file '{x}'\n" for x in Path.cwd().glob("*.wav")),
    text=True, capture_output=True)

Relying on /dev/stdin for the input file is somewhat platform-dependent; in the worst case, you'll need to refactor to use a temporary file, or fall back to using the shell after all.

subprocess.run(r"""ffmpeg -f concat -safe 0 -i <(printf "file '%s'\n" $PWD/*.wav) -c copy output.wav""",
    shell=True, executable='/bin/bash',
    text=True, capture_output=True)

As noted in comments, you should either use shell=True and pass in a single string as the first argument for the shell to parse, or else pass in a list of tokens without shell=True and with no shell features like wildcard expansion, command substitution, variable interpolation, redirection, shell builtins, etc etc.

If you really wanted to explicitly run Bash, the syntax for that would look like

subprocess.run(
    ['bash', '-c',
     r"""ffmpeg -f concat -safe 0 -i <(printf "file '%s'\n" $PWD/*.wav) -c copy output.wav"""],
    text=True, capture_output=True)

(The syntax bash for loop etc tries to find a file named for and run it with Bash, passing in loop and etc as arguments.)

It's not clear why you are using capture_output=True here; in order for that to be useful, you need to examine the .stdout (and/or perhaps .stderr) attributes of the object returned by subprocess.run. If you just want to discard the output, use stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL

edited Dec 4, 2022 at 10:40

answered Dec 4, 2022 at 9:18

tripleee

192k37 gold badges318 silver badges367 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

tripleee Over a year ago

For more, perhaps see my top-voted answer and/or Actual meaning of shell=True in subprocess

tripleee Over a year ago

If ffmpeg supports ffmpeg -i - for reading from standard input, using - instead of /dev/stdin should be portable and robust. Alas, I'm not in a place where I can test this. (You could then use printf "file '%s'\n" $PWD/*.wav | ffmpeg -i - to avoid the Bash process substitution syntax; this should work with any sh.)

555Russich Over a year ago

thanks a lot! I red your 3 answers about subprocess and most likely will read them again

Collectives™ on Stack Overflow

Error while executing bash for loop from python subprocess

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related