Queue between Python2 and Python3

Question

I have a very simple queue which is used between Python3 modules in Raspberry Pi. I want to send messages to the same queue from one Python2 module. Does not work.

Server is running in Python3 as in this example:

from multiprocessing.managers import BaseManager
import Queue
queue = Queue.Queue()
class QueueManager(BaseManager): pass
QueueManager.register('get_queue', callable=lambda:queue)
m = QueueManager(address=('', 50000), authkey='abracadabra')
s = m.get_server()
s.serve_forever()

Sender is like this:

from multiprocessing.managers import BaseManager
class QueueManager(BaseManager): pass
QueueManager.register('get_queue')
m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra')
m.connect()
queue = m.get_queue()
queue.put('hello')

With both only in Python3 or in Python2 everything runs perfectly. If server is in Python3 and sender is in Python2, result is the following error:

Blockquote

Traceback (most recent call last): File "sender_V2.py", line 22, in m.connect() File "/usr/lib/python2.7/multiprocessing/managers.py", line 501, in connect >dispatch(conn, None, 'dummy') File "/usr/lib/python2.7/multiprocessing/managers.py", line 102, in dispatch >kind, result = c.recv() ValueError: unsupported pickle protocol: 3

Python 2 can't deserialise data pickled by Python 3 using the latest protocol, see e.g. docs.python.org/3/library/pickle.html#data-stream-format. There's a workaround here, but you might be better using a proper external queue where you can control the serialisation format. See also stackoverflow.com/q/54135972/3001761 — jonrsharpe
– jonrsharpe, Commented Jan 20, 2019 at 15:30
It might be easier to just use language agnostic MQ like ZeroMQ, Protobufs, or even just simple JSON over UDP. — Gillespie
– Gillespie, Commented Jan 20, 2019 at 18:54
Unfortunately changing the existing solution to something new is too much work. It is surprising that compatibility mode between between Python 2 and Python 3 is not available. Should not be too difficult, because the same code using queue works exactly the same way in both pythons. Naturally with the known syntax changes. — Juhani Miettunen
– Juhani Miettunen, Commented Jan 21, 2019 at 6:55
This is the easiest way to go: stackoverflow.com/questions/27863832/… — Juhani Miettunen
– Juhani Miettunen, Commented Jan 22, 2019 at 10:36

Joab.Ai · Accepted Answer · 2023-01-15 01:53:24Z

0

See this Github link

I used REDIS (with the server running on Ubuntu under Windows WSL) and I was able to pass a Pandas DataFrame from a Py2 miniconda shell to a Py3 miniconda shell running different scripts. The Py2 was running hardware where the company had not upgraded drivers and such to Py3. But the GUI and UX were best done in Py3. REDIS is pretty cool but it was kind of tricky since Py2 needed to use the BytesIO instead of StringIO wrapper that would normally work in Py3.

answered Jan 15, 2023 at 1:53

Joab.Ai

133 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Community Over a year ago

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

Aero Blue Over a year ago

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review

Collectives™ on Stack Overflow

Queue between Python2 and Python3

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related