converting bash scripts to Python - how to handle heredocs?

Question

I'm learning Python while converting some bash scripts to Python shell scripts. One thing I don't yet understand is how to deal with the heredocs used in these scripts. Here are two examples of how the bash scripts use heredocs:

The most important thing I need to know how to do in Python is this first case where the heredoc is used to provide standard responses to commands so the command can run non-interactively:

sudo command << 'EOF'
prompt_response1
    prompt_response2
EOF

Second, tee is used like to this to create a file for which sudo permissions are required:

sudo tee /etc/xdg/autostart/updateNotificationChecker.desktop > /dev/null << 'EOF' 
[Desktop Entry]
Name=Update Notification
Exec=bash /usr/local/bin/updateNotification.sh
Terminal=false
Type=Application
NoDisplay=true
EOF

How would I do these things in Python?

i have to recommend that you learn some pythonic way to perform the actions described by sudo command <<'EOF' as opposed to launching a subprocess as would be done in bash. likely, there's a module or library available that you'd like to explore for that situation, so you could do it in pure python instead of fork() out or god-forbid system() — Josh McGee
– Josh McGee, Commented Jul 24, 2013 at 21:27
that sounds like good advice... just need to know how to do it now. — MountainX
– MountainX, Commented Jul 25, 2013 at 0:43
that's just a matter of learning the language and the tools it provides; the only problem i could imagine would be elevating to root permissions. it's not very nice to jump to root in the middle of a program. — Josh McGee
– Josh McGee, Commented Jul 25, 2013 at 7:11
For doing things like sudo I think shell scripting is best. Such things are hard in Python and when you use system() or popen2 you result in an unnecessarily complicated program. There's things better done in the shell than in Python. On the issue of heredocs, in Python you can use multiline strings (using textwrap.dedent() to make them look better). — Antonis Christofides
– Antonis Christofides, Commented Jul 25, 2013 at 9:12

Community · Accepted Answer · 2020-06-20 09:12:55Z

3

Heredoc in Python

Use multiline string (triple-quote string ''' or """). See Strings from tutorial.

Run command

import subprocess
subprocess.Popen(['cat'], stdin=subprocess.PIPE).communicate('''
    Hello multiline-string
    simliar to heredoc.
''')

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Jul 25, 2013 at 9:19

falsetru

371k69 gold badges769 silver badges659 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

MountainX Over a year ago

just to be clear, your "run command" example can be used to pass responses to a command that normally expects interactive input? In other words, I'm looking for a replacement for expect. After asking my question I found that Python has pexpect. But it seems you are showing me that this can be done with stdin=subprocess.PIPE).communicate. True? Your example doesn't make it clear because cat doesn't require any input...

falsetru Over a year ago

@MountainX, cat read input if no command line argument given.

MountainX · Accepted Answer · 2013-07-26 17:42:58Z

sh (previously pbs) is a full-fledged subprocess interface for Python that allows you to call any program as if it were a function:

from sh import ifconfig
print(ifconfig("wlan0"))

Full docs: http://amoffat.github.com/sh
Follow on Github: http://github.com/amoffat/sh

Example of how it can solve the first problem from this question:

from sh import ssh
import os, sys

# open stdout in unbuffered mode
sys.stdout = os.fdopen(sys.stdout.fileno(), "wb", 0)

aggregated = ""
def ssh_interact(char, stdin):
    global aggregated
    sys.stdout.write(char.encode())
    aggregated += char
    if aggregated.endswith("password: "):
        stdin.put("correcthorsebatterystaple\n")

p = ssh("10.10.10.100", _out=ssh_interact, _out_bufsize=0, _tty_in=True)
p.wait()

It can handle sudo like this:

with sudo:
    print(ls("/root"))

It has a neat feature called STDOUT/ERR callbacks:

sh can use callbacks to process output incrementally. This is done much like redirection: by passing an argument to either the _out or _err (or both) special keyword arguments, except this time, you pass a callable. This callable will be called for each line (or chunk) of data that your command outputs.

Finally, as part of the standard Python tools there is raw_input which writes to standard output and reads from standard input. That will also solve the second problem in this question.

Collectives™ on Stack Overflow

converting bash scripts to Python - how to handle heredocs?

2 Answers 2

Heredoc in Python

Run command

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Heredoc in Python

Run command

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related