Using stdin instead of a file in a python call to a perl script

Question

I'm running a perl script that accepts a file as input from Python using subprocess.Popen(). I now need the input to the script to accept input from the standard input and not a file. If I run the perl scrip from the shell like this:

perl thescript.perl --in /dev/stdin --other_args other_values

It works perfectly. However, in python, nothing happens using the following commands:

mytext = "hi there"
args = ["perl", "myscript.perl", "--in", "/dev/stdin", "--other_args", other_values]
pipe = subprocess.Popen(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
result = pipe.communicate(input=mytext.encode("utf8"))[0]`

result always returns empty (I've also tried using pipe.stdin.write(mytext") and result=pipe.stdout.read())

Please let me know what I'm doing wrong.

1- are there shell meta-characters such as spaces in other_values? Is it a single parameter? 2- Does echo hi there | perl thescript.pl --in /dev/stdin ... work? — jfs
– jfs, Commented Apr 6, 2016 at 22:43
I dunno what is wrong but I'd try either stderr=subprocess.STDOUT: or stderr=subprocess.PIPE so you will also see the errors that are happening. — odedsh
– odedsh, Commented Apr 7, 2016 at 15:08
@J.F.Sebastian, 1- other_values don't have any meta-characters. 2- Yes, in the command line with echo works. — Shani Shalgi
– Shani Shalgi, Commented Apr 10, 2016 at 9:03
@odedsh, I added stderr but there is no output there either. — Shani Shalgi
– Shani Shalgi, Commented Apr 10, 2016 at 9:03

Shani Shalgi · Accepted Answer · 2016-04-10 13:29:21Z

2

Thanks to the comments by @J.F.Sebastian above, I managed to solve this problem with echo and pipes.

args = ["perl", "myscript.perl", "--in", "/dev/stdin", "other_args", other_vals]
pipe1 = subprocess.Popen(["echo", mytext], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
pipe2 = subprocess.Popen(args, stdin=pipe1.stdout, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
pipe1.stdout.close()
result = pipe2.communicate()[0]

Which returns the expected output. Still not sure why the original (posted in the question) didn't work (using communicate to send the text to the stdin)

answered Apr 10, 2016 at 13:29

Shani Shalgi

7151 gold badge7 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

jfs · Accepted Answer · 2016-04-06 22:43:05Z

0

/dev/stdin should work (if it works in the shell on your system):

>>> from subprocess import Popen, PIPE
>>> import sys
>>> p = Popen([sys.executable, '-c', 'print(open("/dev/stdin").read()[::-1])'],
...           stdin=PIPE, stdout=PIPE)
>>> p.communicate(b'ab')[0]
'ba\n'

stdin=PIPE creates a pipe and connects it to the child process' stdin. Reading from /dev/stdin is equivalent to reading from the standard input (0 fd) and therefore the child reads from the pipe here as shown in the example.

answered Apr 6, 2016 at 22:43

jfs

417k210 gold badges1k silver badges1.7k bronze badges

8 Comments

Shani Shalgi Over a year ago

Your example works in python, but my Perl script accepts --in <filename> as input, so I'm not sure how this will work in my case. I tried subprocess.Popen(["perl", "myscript.pl", "--in", "/dev/stdin"], stdin=subprocess.PIPE, stdout=subprocess.PIPE) which doesn't work (communicate returns no error but no output)

jfs Over a year ago

@ShaniShalgi do you see open("/dev/stdin").read() in the answer? What do you think it does? Are you sure subprocess.check_call("each input | perl ... --in /dev/stdin >output", shell=True) works? (Do you see the expected output in the output file?)

Shani Shalgi Over a year ago

I'm not sure we understand each other. In my question I wrote that --in "/dev/stdin" doesn't work in python. subprocess.check_call does not return, subprocess.check_output returns nothing exactly like Popen+communicate. Also, the --in parameter does not accept open("/dev/stdin").read(), only strings. Can you give me the example you mean using the myperl.pl annotation?

jfs Over a year ago

@ShaniShalgi ok. Let's do one small step at a time: do you see the expected output in the output file if you run: subprocess.check_call("echo hi there | perl thescript.perl --in /dev/stdin >output", shell=True)?

Shani Shalgi Over a year ago

Yes this works! I now solved the problem using:

pipe1 = subprocess.Popen(["echo", sentence], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) pipe2 = subprocess.Popen(["perl", "myscript.perl", "--in", "/dev/stdin"], stdin=pipe1.stdout, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) pipe1.stdout.close() result = pipe2.communicate()[0]

Thank you for your patience and help

|

Collectives™ on Stack Overflow

Using stdin instead of a file in a python call to a perl script

2 Answers 2

Comments

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related