Python shared queue - 2 different threads

Question

import time
from flask import Flask, jsonify
from multiprocessing import Process, Value

app = Flask(__name__)

class Queue:
    def __init__(self):
        self.items = []

    def isEmpty(self):
        return self.items == []

    def enqueue(self, item):
        self.items.insert(0,item)

    def dequeue(self):
        return self.items.pop()

    def size(self):
        return len(self.items)

tasks = [
   {
      'id': 1,
      'title': u'Buy groceries',
      'description': u'Milk, Cheese, Pizza, Fruit, Tylenol',
      'done': False
   },
   {
      'id': 2,
      'title': u'Learn Python',
      'description': u'Need to find a good Python tutorial on the web',
      'done': False
   }
]

q = Queue()

@app.route('/todo/api/v1.0/tasks', methods=['GET'])
def get_tasks():

   q.enqueue('cat')
   print("Size: " + str(q.size()))

   return jsonify({'tasks': tasks})


def record_loop(loop_on):
   while True:
      if loop_on.value == True:
         print("loop running")

         q.enqueue('dog')
         print("Size: " + str(q.size()))

      time.sleep(1)


if __name__ == "__main__":
   recording_on = Value('b', True)
   p = Process(target=record_loop, args=(recording_on,))
   p.start()
   app.run(debug=True, use_reloader=False)
   p.join()

I have a global class called queue. The queue's data is shared between 2 different functions, but its not working.

Why is the queue in the function 'get_tasks()' size always equal 1? I thought queues are thread safe and they can be shared between different processes?

The size is not always 1 for me. It's incrementing every time record_loop runs. What is the end goal? — Alex W
– Alex W, Commented Mar 6, 2018 at 0:30
Make a task queue that is shared between these two functions. They need to have access to the same queue. Why isn't it changing the size in the 'get_tasks()' function? — TommySmithers
– TommySmithers, Commented Mar 6, 2018 at 0:31

Brendan Abel · Accepted Answer · 2018-03-06 00:38:33Z

4

When you create a new process, it copies the current environment and re-creates it in the new environment. Once you create the process, those global variables are no longer shared.

However, the multiprocessing library has included ways to communicate between processes

You can still use a queue, but you'll have to use the multiprocessing Queue instead of your home-rolled one. Under the hood, each process has its own Queue object, and they're simply piping information back and forth between the processes to sync the queue state.

answered Mar 6, 2018 at 0:38

Brendan Abel

38k16 gold badges101 silver badges126 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python shared queue - 2 different threads

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related