run piped bash commnads from python

Question

I want to run the following bash command from within my python script

tail input.txt | grep <pattern>

I have written the following lines

bashCommand = "tail input.txt | grep <pattern>'"
process = subprocess.Popen(bashCommand.split(), stdout=subprocess.PIPE)

But this ends up simply printing out the tail of the input file and not the pattern I am trying to grep. How do I circumvent this?

Robbe VdD · Accepted Answer · 2020-11-10 18:46:01Z

1

You can pass shell=True to subprocess.Popen. This will run the command through the shell. If you do this you will need to pass a string instead of an list:

process = subprocess.Popen("tail input.txt | grep ", stdout=subprocess.PIPE, shell=True) print process.communicate()`

You can find a more detailed explanation here: https://unix.stackexchange.com/questions/282839/why-wont-this-bash-command-run-when-called-by-python

answered Nov 10, 2020 at 18:46

Robbe VdD

262 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

pshah Over a year ago

Thanks Robbe! Is there a way to make it work with a variable? Where <pattern> can be substituted with a var?

Robbe VdD Over a year ago

Yes, you can always use string formatting. Or you can use the solution that chepner came up with. His solutions is more robust.

chepner · Accepted Answer · 2020-11-10 19:29:09Z

1

Consider implementing the pipe in Python, rather than the shell.

from subprocess import Popen, PIPE
p1 = Popen(["tail", "input.txt"], stdout=PIPE)
process = Popen(["grep", "<pattern>"], stdin=p1.stdout)

answered Nov 10, 2020 at 19:29

chepner

538k77 gold badges594 silver badges746 bronze badges

Collectives™ on Stack Overflow

run piped bash commnads from python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related