1
diff -u <(echo "aba"| fold -w1) <(echo "abaa" | fold -w1)

I want to execute the above command from Python code. Below is the code I have written.

cmd = "diff -u < (echo 'aba'| fold -w1) < (echo 'abaa' | fold -w1)"
os.system(cmd)

Running the above code, I get sh: 1: Syntax error: "(" unexpected error. As per my knowledge the unix os can't parse the echo command which is inside the braces.

Help me fix this error.

3 Answers 3

3

First, os.system() is discouraged in favor of subprocess.call(cmd, shell=True). That's worth knowing because there's a lot of additional detail in the subprocess documentation, including this description of the shell=True parameter (emphasis added):

On POSIX with shell=True, the shell defaults to /bin/sh.... Popen does the equivalent of:

Popen(['/bin/sh', '-c', args[0], args[1], ...])

So now we know why your command doesn't work - it's not invoking Bash. As mhawke suggests you should instead invoke bash directly, but you should prefer the subprocess module over os.system():

>>> subprocess.call(['/bin/bash', '-c', 'diff -u <(echo "aba"| fold -w1) <(echo "abaa" | fold -w1)'])
--- /dev/fd/63  2017-02-25 14:32:49.000000000 -0800
+++ /dev/fd/62  2017-02-25 14:32:49.000000000 -0800
@@ -1,3 +1,4 @@
 a
 b
 a
+a
1

Note that, since we're explicitly invoking the Bash shell, we don't need shell=True, and since the command we're telling Bash to invoke is a single argument we don't need to repeatedly escape them, e.g. with """ as mhawke did.

Once you've verified this command works, you'll likely want to move away from simply invoking call() to one of subprocess other functions that are more scripting-friendly, such as run(), which returns a CompletedProcess object you can inspect.

As Jean-François Fabre suggests you can do a lot more powerful things with subprocess as well, including starting the <() substitutions as separate processes and piping them into a call to diff, thus avoiding needing to invoke bash or write Bash syntax in Python. It's more verbose, but more extensible and maintainable.

Sign up to request clarification or add additional context in comments.

1 Comment

and portable too, you'd only have to install diff & fold on windows.
2

The command runs fine in bash, however, os.system() is executing the command in /bin/sh. You can check with:

>>> os.system('echo $0')
sh
0

The command fails when executed with /bin/sh:

[mhawke@localhost-localdomain ~]$ /bin/sh
sh-4.3$ diff -u <(echo "aba"| fold -w1) <(echo "abaa" | fold -w1)
sh: syntax error near unexpected token `('
sh-4.3$ 

You can explicitly run the command in bash like this:

>>> os.system("""bash -c 'diff -u <(echo "aba"| fold -w1) <(echo "abaa" | fold -w1)'""")
--- /dev/fd/63  2017-02-26 09:18:14.633395225 +1100
+++ /dev/fd/62  2017-02-26 09:18:14.633395225 +1100
@@ -1,3 +1,4 @@
 a
 b
 a
+a
256

Since you are probably interested in the output of the command you would ordinarily be able to use subprocess.check_output() to execute the command and collect its output. Unfortunately diff likes to return non-zero exit codes when it detects differences in the input files, so that prevents simply using check_output. You can cheat by piping diff's output through cat:

>>> from subprocess import check_output
>>> output = check_output(['bash', '-c', 'diff -u <(echo "aba"| fold -w1) <(echo "abaa" | fold -w1) | cat'])
>>> print(output)
b'--- /dev/fd/63\t2017-02-26 10:02:56.814044987 +1100\n+++ /dev/fd/62\t2017-02-26 10:02:56.814044987 +1100\n@@ -1,3 +1,4 @@\n a\n b\n a\n+a\n'

>>> print(str(output, encoding='utf8'))
--- /dev/fd/63  2017-02-26 10:02:56.814044987 +1100
+++ /dev/fd/62  2017-02-26 10:02:56.814044987 +1100
@@ -1,3 +1,4 @@
 a
 b
 a
+a

4 Comments

the best way would be to use 3 Popen objects chained together with pipes.
@Jean-FrançoisFabre: that's true, but it depends on your point of view. OP is asking about os.system().
I was not criticizing your answer but the question :) the +1 is mine. I should have commented the question, not the answer. I can't blame you for not trying to write the 3 pipes in python!
@Jean-FrançoisFabre: no problem, I didn't take it as criticism - thanks for the upvote.
0

or you could

import subprocess
cmd = """bash -c 'diff -u <(echo "aba"| fold -w1) <(echo "abaa" | fold -w1)'"""

ps = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = ps.communicate()[0]

2 Comments

what's the point of using subprocess instead of os.system() if you're not going all the way with 3 subprocess objects and drop the shell=True part?
@Jean-FrançoisFabre "the best way would be to use 3 Popen objects chained together with pipes." is this what you mean?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.