2

I am trying to run the shell command

echo -e 'FROM busybox\nRUN echo "hello world"' | docker build -t myimage:latest -

from jupyter notebook using subprocesses

I have tried the code

p = subprocess.Popen('''echo -e 'FROM busybox\nRUN echo "hello world"' | docker build -t myimage:latest - ''', shell=True)
p.communicate()

and some iterations with run() or call(), but everytime the output is

-e 'FROM busybox

It seems that the new line character \n causes the problem. Any ideas to solve the problem?

1
  • Create sh script file, put all your lines there and run it from python. Commented Jul 18, 2021 at 19:54

1 Answer 1

4

The \n gets parsed by Python into a literal newline. You can avoid that by using a raw string instead,

p = subprocess.run(
    r'''echo -e 'FROM busybox\nRUN echo "hello world"' | docker build -t myimage:latest - ''',
    shell=True, check=True)

but I would recommend running a single process and passing in the output from Python; this also avoids a shell, which is generally desirable.

p = subprocess.run(['docker', 'build', '-t', 'myimage:latest', '-'],
    input='FROM busybox\nRUN echo "hello world"',
    text=True, check=True)

Notice also how we prefer subprocess.run() over the more primitive subprocess.Popen(); as suggested in the documentation, you want to avoid this low-level function whenever you can. With check=True we also take care to propagate any subprocess errors up to the Python parent process.

As an aside, printf is both more versatile and more portable than echo -e; I would generally recommend you to avoid echo -e altogether.

This ideone demo with nl instead of docker build demonstrates the variations, and coincidentally proves why you want to avoid echo -e even if your login shell is e.g. Bash (in which case you'd think it should be supported; but subprocess doesn't use your login shell).

Sign up to request clarification or add additional context in comments.

3 Comments

You could add that to get a raw string one must add r in front of that string as such: r'string goes here' (although that is basic doc info and should be easily found)
Thank you for the great answer. The second solution with run() executed successfully with the addition of encode() to the "input", because a byte-like object is required, not a string. The problem with the raw string is that \n is passed as a literal to echo and it's not interpreted as new line.
Gotcha, sorry about that, and thanks for the edit!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.