3

I'm relatively to to python and I'm trying to write a python script to which one can pipe the output of a command or another script.

example command | python_sript.py

In python script I'll basically analyze the output of command and save it to file.

I thought I'll be able to do this with redirecting sys.stdin to subprocess.PIPE, but it didn't work.

sys.stdin = subprocess.PIPE

Can some one please suggest how should I approach this?

Also what would happen if command or script pipes data at faster rate then what python script can process.

Note: When I use this script

import sys
data = sys.stdin.readline()
print(data)

I get this

D:\>echo "test" | tee.py
The process tried to write to a nonexistent pipe.
Traceback (most recent call last):
  File "D:\Profiles\Administrator\My Documents\Workspace\tee.py", line 3, in <mo
dule>
    data = sys.stdin.readline()
AttributeError: 'NoneType' object has no attribute 'readline'

and when I use this

import sys
data = input()
print(data)

I get this

D:\>echo "test" | tee.py
The process tried to write to a nonexistent pipe.
Traceback (most recent call last):
  File "D:\Profiles\Administrator\My Documents\Workspace\tee.py", line 3, in <mo
dule>
    data = input()
RuntimeError: input(): lost sys.stdin
1
  • In your edit it looks like you have done something silly with sys.stdin. Try sys.__stdin__. Commented Nov 16, 2010 at 18:33

3 Answers 3

4

On (old) Windows when you use pipes you need to call python scripts using python executable by default due to NT redirection bug:

D:\> echo "test" | python tee.py

After that sys.stdin, raw_input(), fileinput.input() should work as expected.

Sign up to request clarification or add additional context in comments.

Comments

1

You seem to be a bit unclear on how pipes work. They are handled by the OS, not the individual programs -- so if you write a Python script designed to take input data from a pipe, it should just read stdin as normal and the redirection will be handled for you. The data will be consumed as fast as they are generated; if the script consumes data more slowly than they are generated then they will be stored in a buffer.

Or are you trying to communicate between two Python scripts? If so there are better ways than through stdin and stdout.

2 Comments

Which better ways would you recommend?
@Thomas: 1) Put the data handler in a module and import it. 2) Spawn a separate process and pipe it.
0

Question 1 (reading from a pipe) has been answered by others: Just read stdin in python_script.py. The OS handles the redirection through pipes.

Question 2 (writing/reading speed) has also been answered: OS pipes are buffered, so nothing to be concerned about here.

Question 3 (the stack traces): Your first program runs fine with me. - Are you really giving the whole code here?

As for subprocess.PIPE: You really only want to use this, if you start a new command from within your running Python script, so you can communicate with the child process. As you are not doing this, it is of no use to you. Pipes on the shell level don't constitute parent-child processes.

1 Comment

About the stack trace, first program is not working for me. Is sys.stdin.readline() not supported under Python 3.1? And yes these are the complete programs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.