Run python commands in a docker container from python script on host

Question

I have a docker image and and associated container than runs a jupyter-lab server. On this docker image I have a very specific python module that cannot be installed on the host. On my host, I have all my work environment that I don't want to run on the docker container.

I would like to use that module from python script running on the host. My first idea is to use docker-py (https://github.com/docker/docker-py) on the host like this:

import docker
client = docker.from_env()
container = client.container.run("myImage", detach=True)
container.exec_run("python -c 'import mymodule; # do stuff; print(something)'")

and get the output and keep working in my script.

Is there a better solution? Is there a way to connect to the jupyter server within the script on the host for example?

Thanks

I'm not very familiar with jupyter, but can't you just do port-mapping with docker and than use that server as if it was running on 127.0.0.1 ? — SMoenig
– SMoenig, Commented Jun 23, 2021 at 10:53
It is also related to this discussion (4 years ago) stackoverflow.com/questions/39091186/… — dagnic
– dagnic, Commented Jun 23, 2021 at 11:20

MikZuit · Accepted Answer · 2021-06-27 10:48:23Z

0

First. As @dagnic states on his comment there are those 2 modules that let you execute docker runtime in you python script (there's probably more, another different question would be "which one is best").
Second. Without knowing anything about Jupiter, but since you call it "server" , it would mean to me that you are able to port mapping that server (remember -p 8080:80 or --publish 8080:80, yeah that's it!). after setting a port mapping for your container you would be able to ie use pycurl module an "talk" to that service.

Remember, if you "talk on a port" to your server, you might also want to do this using i.e with docker-py.

Since you asked if any better solution exists: This two method would be the more popular. First one would convenient for your script, second would launch a server and you can use pycurl from your host script as you asked (connect to the jupyter server) .ie if you launch jupyter server like:

docker run -p 9999:8888 -it -e JUPYTER_ENABLE_LAB=yes jupyter/base-notebook:latest

you can pycurl like:

import pycurl
from io import BytesIO 

b_obj = BytesIO() 
crl = pycurl.Curl() 

# Set URL value
crl.setopt(crl.URL, 'https://localhost:8888')

# Write bytes that are utf-8 encoded
crl.setopt(crl.WRITEDATA, b_obj)

# Perform a file transfer 
crl.perform() 

# End curl session
crl.close()

# Get the content stored in the BytesIO object (in byte characters) 
get_body = b_obj.getvalue()

# Decode the bytes stored in get_body to HTML and print the result 
print('Output of GET request:\n%s' % get_body.decode('utf8'))

Update:

So you have two questions:

1. Is there a better solution? Basically using docker-py module and run jupyter server in a docker container (and a few other options not involving docker I suppose)

2. Is there a way to connect to the jupyter server within the script on the host for example?

Here, there is an example how to run jupyter in docker. enter link description here

The rest is use pycurl from your code to talk to that jupyther server from your host computer.

edited Jun 27, 2021 at 10:48

answered Jun 23, 2021 at 11:52

MikZuit

7441 gold badge6 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

dagnic Over a year ago

Thanks. In your pycurl example, how to use the python module that lies in the docker container?

MikZuit Over a year ago

you don't have to use python docker module fi you use pycurl , in this case you are just "chatting" with https server ( juypyter ) which is launch in a container and port exposed in your local computer

dagnic Over a year ago

I still don't understand how this can help me. Can you give me a more detailed example of what I can do with this?

MikZuit Over a year ago

you should create another question or improve this one with: which port your jupyter server is (this would let us know if you dont have any other problem running this docker container) and provide your code with pycurl in order to check any issues in there. Check the last link i provided u in answer

Collectives™ on Stack Overflow

Run python commands in a docker container from python script on host

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related