Python subprocess using perl for formatting is giving incomplete output

Question

I'm having an issue reading output from a python subprocess command.

The bash command from whose output I want to read:

pacmd list-sink-inputs | tr '\n' '\r' | perl -pe 's/ *index: ([0-9]+).+?application\.process\.id = "([^\r]+)"\r.+?(?=index:|$)/\2:\1\r/g' | tr '\r' '\n'

When I run this via bash I get the intended output:

4 sink input(s) available.
6249:72
20341:84
20344:86
20350:87

When I try to get it's output via python's subprocess running either one :

subprocess.Popen(cmnd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()[0].decode('UTF-8')
check_output(cmnd,shell=True).decode('UTF-8')
subprocess.run(cmnd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.decode('utf-8')

where cmnd = """pacmd list-sink-inputs | tr '\n' '\r' | perl -pe 's/ *index: ([0-9]+).+?application\.process\.id = "([^\r]+)"\r.+?(?=index:|$)/\2:\1\r/g' | tr '\r' '\n'"""

It gives the following output:

'4 sink input(s) available.\n\x02:\x01\n\x02:\x01\n\x02:\x01\n\x02:\x01\n'

Which is unintended as it doesn't have the 6249:72 ,etc. numbers I want. Even stderr is blank and returncode is 0 as intended.

The only workaround, I could find was to redirect the bash output to a text file and then read the text file via python which I don't want to use because that's unnecessary file IO.

I've already gone through Missing output from subprocess command, Python Subprocess Grep, Python subprocess run() is giving abnormal output [duplicate] and many others but can't wrap my head around what's going wrong.

ikegami · Accepted Answer · 2021-07-10 16:07:27Z

2

You have a quoting issue. """\1""" means chr(0o1). To produce the string \1, you could use """\\1""". The other instances of \ should be \\ as well.

Since all instances of \ need to be escaped, you could also use r"""\1""".

Other issues:

\1 and \2 outside of a regular expression is wrong anyways. You should be using $1 and $2.
There's no use for a mutliline literal here. "..." or r"..." would suffice.
The whole tr business can be avoided by using -0777 to cause perl to treat the entire file as one line.

This gives us:

cmnd = "pacmd list-sink-inputs | perl -0777pe's/ *index: (\\d+).+?application\\.process\\.id = "([^\\n]+)"\\n.+?(?=index:|$)/$2:$1\\n/sag'"

or

cmnd = r"pacmd list-sink-inputs | perl -0777pe's/ *index: (\d+).+?application\.process\.id = "([^\n]+)"\n.+?(?=index:|$)/$2:$1\n/sag'"

But why is Perl being used at all here? You could easily do the same thing in Python!

edited Jul 10, 2021 at 16:07

answered Jul 10, 2021 at 16:01

ikegami

391k17 gold badges291 silver badges555 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Atharva Over a year ago

ohh, got the issue, Thanks a lot. Your answer works after using triple quotes: cmnd = r"""bash-command""" worked. As to why I'm using Perl within the bash command and not processing the output in Python, Perl is way faster than Python for text processing. I tried but coudn't upvote your answer because of the min. 15 rep limit.

Collectives™ on Stack Overflow

Python subprocess using perl for formatting is giving incomplete output

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related