How to pass socket variable while using exec() in Python?

Question

I have a program version A that calls execve() to execute another program version B while retaining the same PID.

This program A has a socket object and we would like the other program B to access the same socket to send some data to the established connection.

The problem is that I haven't found a way to successfully recover the serialized socket, I'm saving it to a file, opening the file again in the new program B but it is not the same object, is uninitialized and doesn't have any attributes set, I have used dill and pickle.

Is there a safe way to pass variable objects while calling exec()? Or a way to get the pointer reference of the object and pass that pointer forward?

Version A:

#!/bin/env python
import os
import dill as pickle
import socket

print "Version A"
print os.getpid()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

s.connect(("127.0.0.1",80))

with open('file1.pickle', 'wb') as handle:
    pickle.dump(s, handle)

os.execlp('python', 'python', 'verB.py')

Version B:

#!/bin/env python
import os,pdb,sys
import dill as pickle
import socket

print "Version B"
print os.getpid()
infile = open('file1.pickle','rb')
infile.seek(0, 0)
s = pickle.load(infile)
infile.close()

s.send("mymessage")

s.close()

Version B socket s is uninitialized, how to recover the socket?

Python doesn't have pointers. Why are you using exec? You should design your other program to be importable, that is the sanest way of accomplish this. Then you can just do import other_program; whatever_we_need = other_program.main() — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Apr 9, 2020 at 20:06
No. Python doesn't have pointers. You are describing implementation details of CPython, that are neither here nor there, because python the language doesn't have pointers. If you disagree, show me how to use a pointer, dereference a pointer, do some pointer arithmetic. I think you'll find you can't. And you don't copy lists into variables. That isn't the semantics of Python. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Apr 9, 2020 at 20:09
... right, so again, why don't you use the one program as a module, then state is shared automatically. It is with exec as well. So it isn't exactly clear the exact issue you are running in to. It would help if you gave an example. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Apr 9, 2020 at 20:11
You can't pass the socket to the exec'd process; but you can pass the fd number, and then open it with socket.socket(fileno=number). Probably, if python's exec does not close fds from the parent when called. — vanza
– vanza, Commented Apr 9, 2020 at 20:37
Aha! See, this is critical, why you need to provide a minimal reproducible example. You kept saying exec so I assumed you meant the built-in exec. Yes, as @vanza says, you can pass the fileno arg, and os.exec* shouldn't close it. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Apr 9, 2020 at 20:55

tdelaney · Accepted Answer · 2020-04-09 21:23:40Z

On unix-like systems execve programs retain open file descriptors that are not marked FD_CLOEXEC. You need to pass the file descriptor (a small integer) to the new process. Python sets file descriptors to close-on-exec by default, so if your parent is a python process, it needs to specifically change that flag. You can write the socket fd to a file:

open("file1.txt", "w").write(str(s.fileno()))

but more commonly, one would either pass it on the command line or tuck it into the environment. As an example of python calling python with the socket passed on the command line:

verA.py

import socket
import os
import sys

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(("stackoverflow.com", 80))
os.set_inheritable(sock.fileno(), True)
os.execlp(sys.executable, "python", "verB.py", str(sock.fileno()))

verB.py

import socket
import sys

sock = socket.fromfd(int(sys.argv[1]), socket.AF_INET, socket.SOCK_STREAM)
sock.send(b"GET / HTTP/1.1\r\nHost:stackoverflow.com\r\n\r\n")
print(sock.recv(500))

Collectives™ on Stack Overflow

How to pass socket variable while using exec() in Python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related