Running Bash scripts in iPython

Question

I wrote a bash script, run.sh which has a python command with multiple options -

python train.py --lr 0.01 \
            --momentum 0.5 \
            --num_hidden 3 \
            --sizes 100,100,100 \
            --activation sigmoid \
            --loss sq \
            --opt adam \
            --batch_size 20 \
            --anneal true

I tried running this command in iPython -

!./run.sh

However, in iPython I'm not able to access the variables of the python script train.py. Is there some way to run the bash script in iPython so that I can access the variables? I don't want to copy paste the above command from the bash script each and every time.

I'm currently using iPython 5.1.0 on macOS Sierra.

Start your cell with %%bash, then you can input shell code. This is called bash cell magic, if you want to google it. — cel
– cel, Commented Feb 6, 2017 at 5:35
Instead of the %%bash magic, suggested by @cel , you could try to use the %%python magic and then put your script inside the cell and run it directly with the parameters you want — Aelius
– Aelius, Commented Nov 11, 2020 at 12:08

mfardal · Accepted Answer · 2017-02-07 20:13:36Z

1

The python process that runs your script train.py and the python process you're using at the ipython command line are two separate processes. It makes sense that one doesn't know about the variables of the other. There is probably some fancy way to connect the two but I suspect from the way you described the problem that it's not worth the work.

Here's an easier way to get access: you could replace python train.py in your script with python -i train.py. This way you will go into interactive mode in the process that runs your script after it is done, and anything defined at the main level will be accessible. You could insert a call to pdb.set_trace() in your script to stop at an arbitrary point.

answered Feb 7, 2017 at 20:13

mfardal

1764 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Running Bash scripts in iPython

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related