Can someone explain why this short bash / python command does not output "hello"
$ echo hello | python - <<END
import sys
for line in sys.stdin:
print line
END
If I save my python script to a file, this command works as expected.
In script.py
import sys
for line in sys.stdin:
print line
END
$ echo "hello" | python script.py
"hello"
The reason it's not working is that you have conflicting redirections. echo | python tries to tie standard input to the pipe from echo, while python - <<HERE tries to tie standard input to the here document. You can't have both. (The here document wins.)
However, there is really no reason to want to pipe the script itself on standard input.
bash$ echo hello | python -c '
> import sys
> for line in sys.stdin:
> print line'
hello
Another solution might be to embed your data in a triple-quoted string within the script itself.
python <<\____
import sys
data = """
hello
so long
"""
for line in data.strip('\n').split('\n'):
print line
____
The newlines between the triple quotes are all literal, but the .strip('\n') removes any from the beginning or the end. (You can't do that if you have significant newlines at the end, of course.)
The backslash before the here-document separator says to not perform any variable interpolation or command substitutions inside the here document. If you want to use those features, take it out (but then of course take care to individually escape any literal dollar signs or backticks in your Python code).
Because python << END redirects stdin to read the program. You can't also read the line piped from echo on stdin.
Oh boy, that makes sense! I've been struggling with this for the past hour and finally I understand, thanks both of you.
To provide a working alternative for the original program, you can do something like this:
$ echo hello | python <(cat <<END
import sys
for line in sys.stdin:
print line
END
)
But be careful! If you put this in a file, the final END need to be touching the left gutter (no whitespaces). Otherwise it won't parse with a 'unexpected end of file while looking for matching character' error or something like that (actually, this took me two more hours to figure out from when I started answering this).
Explaining the code, what is happening is that the <<END ... END combined with the <(cat ...) create a file (in /dev/fd) so that it can be fed to python, as if it were a real file. Quoting this explanation about <(...):
It's process substitution. It feeds the output of the command into a FIFO that can be read from like a normal file.
There is another interesting answer about this topic here.
cat. The shell doesn't need the help of cat in a process substitution to feed the here document to Python, though the construct avoids tying up standard input (the shell basically makes the output from the process substitution available as a named temporary file).
<(< <<END ...), which does not work.