iPython embed() automatically exits when calling script from bash file

Question

I have a data processing pipeline setup that I want to debug. The pipeline consists of a bash script that calls a python script.

I usually use iPython's embed() function for debugging. However, when calling the python script from the bash file, the embed() function is called but immediately exited, without me being able to interfere. When running the same python program directly from the command line I don't observe this kind of behavior. Is this intended behavior or am I doing something wrong?

Python 2.7.6 (default, Oct 26 2016, 20:30:19)
Type "copyright", "credits" or "license" for more information.

IPython 2.4.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]:
Do you really want to exit ([y]/n)?
'follow up code prints here'

Please post the relevant code from the Bash script and the Python script. I was not able to reproduce this by making a bash script that runs a Python script that calls embed(). — elethan
– elethan, Commented Dec 20, 2016 at 14:01

MatrixManAtYrService · Accepted Answer · 2019-12-10 00:17:52Z

2

I can replicate the problem like this:

# test.py
import IPython
import sys
print(sys.stdin.read())
IPython.embed()

# session
❯ echo 'foo' | python test.py
foo

Python 3.6.8 (default, Oct  7 2019, 12:59:55)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.10.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: Do you really want to exit ([y]/n)?

❯ # I didn't quit on purpose, it happened automatically

STDIN is not a TTY, so I'm thinking that IPython is worried that the inbound text (via the pipe) won't be a user typing. It doesn't want foo (from my example above) to spew into the IPython shell and do something unexpected.

You can work around this by getting your terminal id via the tty command, and redirecting stdin to the calling terminal after it has finished reading from the pipe, something like this:

with open('/dev/pts/16') as user_tty:
    sys.stdin=user_tty
    IPython.embed()

For more on ttys, see this post. Note also that if you put the wrong tty in there, input from some other terminal will control IPython.

I'm not sure if it's possible for IPython to know what the calling tty would have been, had it not been overwritten by bash to be the output-side of the pipe.

Edit: Here's my workaround put more simply: How do I debug a script that uses stdin with ipython?

edited Dec 10, 2019 at 0:17

answered Dec 10, 2019 at 0:10

MatrixManAtYrService

9,3992 gold badges58 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

HappyFace Over a year ago

IPython does not seem to be able to read the escape codes correctly when I do this. Things like the tab and arrow keys don't work.

MatrixManAtYrService Over a year ago

You're right @HappyFace, I'm not sure why 2019-me didn't think that was worth mentioning.

Mayank Jaiswal · Accepted Answer · 2016-12-20 16:10:14Z

I ran some experiments to see the behaviour. I noticed that IPython shows the console if any of the ancestor process is terminal.

Following are the files in /tmp directory: x.py

import IPython 
IPython.embed()

call.sh

/usr/bin/python /tmp/x.py

call2.sh

/tmp/call.sh

Experiment 1 Running python x.py does open the IPython shell and waits.

Experiment 2 Running bash call.sh also opens the IPython shell and waits.

Experiment 3 Running bash call2.sh also opens the IPython shell and waits.

As you can see, it does not matter how deep is your IPython.embed call is. It always starts the interactive console and waits.

Lets try if it also works when we fork a new process.

fork.sh

/usr/bin/python /tmp/x.py &

Experiment 4 In this case, IPython shell started but immediately exited. Notice the & at the end. It starts a different process. IPython was not able to access the terminal in this case and hence exited gracefully.

Collectives™ on Stack Overflow

iPython embed() automatically exits when calling script from bash file

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related