retrieving python variables from Cell Magic in Ipython notebook from background subprocess

Question

I'm trying to do a quick and dirty querying of my mongodb database using ipython notebook.

I have several cells each with its own query. Since mongodb can support several connections I would like to run each query in parallel. I thought an ideal way would be just do something like

%%script --bg python
query = pymongo.find(blahbalhba)

You can imagine several cells each with its own query. However I'm not able to access the query returned by pymongo.find.

I understand that this is a subprocess run in a seperate thread, but I have no idea how to access the data since the process is quickly destroyed and the namespace goes away.

I found a similar post for %%bash here but I'm having trouble translating this to a python namespace.

Just M · Accepted Answer · 2014-01-03 14:59:20Z

1

%%script is just a convenient magic, it will not replace writing a full blown magic. The only thing I can see is to write your own magic. Basically if you can do it with a function that takes a string a parameter, you know how to write a magic.

So How would you (like to) write it in pure python ? (Futures, multiprocessing, queuing library ?) ... then move it to a magic.

answered Jan 3, 2014 at 14:59

Just M

28k6 gold badges85 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

jwillis0720 Over a year ago

I think I will use the builtin subprocess and find a way to retrieve the object. One idea is just to stdout into an object, pickle it, and then retrieve it.

Collectives™ on Stack Overflow

retrieving python variables from Cell Magic in Ipython notebook from background subprocess

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related