I'm trying to write a small wsgi application which will put some objects to an external queue after each request. I want to make this in batch, ie. make the webserver put the object to a buffer-like structure in memory, and another thread and/or process for sending these objects to the queue in batch, when buffer is big enough or after certain timeout, and clearing the buffer. I don't want to be in NIH syndrome and not want to bother with threading stuff, however I could not find a suitable code for this job. Any suggestions?
Add a comment
|
3 Answers
Examine https://docs.python.org/library/queue.html to see if it meets your needs.
4 Comments
llimllib
beat me by 16 seconds to the same answer. I think that's what he wants too, but couldn't quite figure it out.
Federico A. Ramponi
Here is what I like most about Python: the 80% of what you're going to implement is already out there. Often, even in the standard library.
S.Lott
And this question has already been asked on SO. stackoverflow.com/questions/394500/…
Tim S.
I know this is a 5-year old thread, but apparently still relevant (since I just found it looking for a buffer-like structure in Python)... I wanted to report that the above link to python.org is no longer valid if anyone cares to update it, if possible. I did find pymotw.com/2/Queue as a possible replacement.
Since you write "thread and/or process", see also multiprocessing.Queue and multiprocessing.JoinableQueue from 2.6. Those are interprocess variants of Queue.
Comments
Use a buffered stream if you are using python 3.0.
