Message queue IPC implementation on Linux

Question

I have a multi-processed perl system that picks up a lot of files ( may be in 10 Millions ) and works on the them one by one

Since each process must pick up exclusive files I need to have one seperate process that does the file-listing and sends the filenames to a message queue

Each worker-process will query the message-queue , get 10 jobs at a time and execute the 10 jobs until the message-queue is empty

What is the best way of implementing a message queue

Gearman, TheSchwartz, Qudo

daxim
– daxim

2011-08-29 08:25:05 +00:00
Commented Aug 29, 2011 at 8:25 — daxim
– daxim, Commented Aug 29, 2011 at 8:25

frezik · Accepted Answer · 2011-08-26 14:17:43Z

1

You could give Queue::Beanstalk a try.

answered Aug 26, 2011 at 14:17

frezik

2,31615 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

daxim · Accepted Answer · 2011-08-29 08:21:44Z

1

I'd vote for Net::RabbitMQ and rabbitmq

edited Aug 29, 2011 at 8:21

daxim

39.3k4 gold badges71 silver badges135 bronze badges

answered Aug 27, 2011 at 4:28

ErikR

52.2k9 gold badges78 silver badges136 bronze badges

Collectives™ on Stack Overflow

Message queue IPC implementation on Linux

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related