How can I share a Queue between processes in Python?

Question

I know there is a Value type could allocate shared memory, but seems it could only contains basic c types.

Is there anyway allow us to pass complex data types like Queue/Map between processes ?

nneonneo · Accepted Answer · 2013-06-13 02:50:42Z

1

Use Managers. Create a Manager object, then invoke manager.Queue (for example) to create a sharable Queue proxy. You can then pickle the returned proxy object and pass that between processes.

answered Jun 13, 2013 at 2:50

nneonneo

181k37 gold badges331 silver badges412 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

WoooHaaaa Over a year ago

Thanks, @nneonneo, How about the Queue's performance if I read and write via multi processes ? Is it using Lock or CAS to make the shared Queue safe ?

nneonneo Over a year ago

The shared Queue is implemented using Queue.Queue, which internally uses locks to achieve thread safety. Process safety is ensured by the Manager, which controls access. You should benchmark the Queue against other approaches and to see how its performance fares on your specific platform.

Collectives™ on Stack Overflow

How can I share a Queue between processes in Python?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related