Python , multithreading with Queue

Question

Let's say I have 2 classes Client and Server

class Client(Thread):

    def __init__(self, serv):
        Thread.__init__(self)
        self.serv = serv

    def run(self):
        self.serv.in.put("Hi")

class Server(Thread):

    def __init__(self):
        Thread.__init__(self)
        self.in = Queue()
        self.out = Queue()


    def run(self):
        while True:
            if self.in.get():
                #smth to do

So. I'm creating Server and Client, after that, Client send initiate message in queue to Server and I can send something back, etc.

However, when there are 2 or 3 Clients connected to 1 Server and using one Queue there could be some problems. For example - Server read information from Client 2 and think that it's initiate message from Client 3, etc.

Thus, I want to create new Thread with in Queue and out Queue for each client and send it back after first message. So now, they would communicate with server from this new thread.

Should I create new class with in and out Queues and send it back to Client?

I've never worked with threads, so I just don't know how to do it. I tried to google it, but found only tutorials about creating threads.

Serge Ballesta · Accepted Answer · 2015-11-29 13:53:22Z

You must think of what are your requirement, what are your objects, what messages are exchanged, and what triggers them.

As far as I understand it, you have one server thread, that waits for a message from a client, and then send something back to this client. That means that the server must identify its client to be able to send the proper message to this client.

My advice is to use one queue for the server, and one queue for each client. When a client sends a message to the server, it also passes a reference to the queue where the answer should be posted back.

Here are minimal changes to your initial code to implement that:

from Queue import Queue from threading import Thread

class Client(Thread):

    def __init__(self, serv):
        Thread.__init__(self)
        self.serv = serv
        self.inq = Queue()   # declares a queue for the client

    def run(self):
        self.serv.inq.put(("Hi", self.inq))  # send a message, and pass along a response address
        resp = self.inq.get()    # get the response
        print("Got: ", resp)


class Server(Thread):

    def __init__(self):
        Thread.__init__(self)
        self.inq = Queue()  # declare a queue for the server

    def run(self):
        while True:
            req = self.inq.get()   # get a message from input queue
            if not req:
                break
            msg, clientq = req    # parse the message
            # other processing
            clientq.put("From serv: " + msg)  #send back a response

That way, Server can serve any number of different clients. But as server itself is not multithreaded, you can only answer one client request at a time.

jamt · Accepted Answer · 2015-11-29 13:35:06Z

If what you want is a server that communicates with each client individually you could use one control queue in the server instance for receiving init messages, and then have in and out queues in each client which are sent to the server through the init message.

As for the risk of mistaking messages, I suggest having a well defined message structure, using class definitions for each type of message.

Something similar to this perhaps:

class Message(object):
    pass

class InitMessage(Message):
    def __init__(self, queue_in, queue_out):
        self.queue_in = queue_in
        self.queue_out = queue_out

class Client(Thread):

    def __init__(self, server_queue, name):
        Thread.__init__(self)
        self.control = server_queue
        self.my_in_queue = Queue()
        self.my_out_queue = Queue()
        self.name = name

    def run(self):
        self.control.put(InitMessage(self.my_in_queue, self.my_out_queue))
        while True:
            if not self.my_in_queue.empty():
                message = self.my_in_queue.get()
                print("Client {name} got message from server of type {type}".format(
                    name=self.name, type=type(message).__name__
                ))
                self.my_in_queue.task_done()
                self.my_out_queue.put(Message())

class Server(Thread):

    def __init__(self):
        Thread.__init__(self)
        self.control = Queue()
        self.clients = {}
        self.id = 0

    def run(self):
        while True:
            while not self.control.empty():
                message = self.control.get()
                if isinstance(message, InitMessage):
                    self.clients[self.id] = {'to_client': message.queue_in,
                                             'from_client': message.queue_out}
                    message.queue_in.put(Message())
                    self.id += 1
                self.control.task_done()

            for client in self.clients:
                if not self.clients[client]['from_client'].empty():
                    message = self.clients[client]['from_client'].get()
                    print("got message from client {id}".format(id=client))
                    self.clients[client]['from_client'].task_done()
                    self.clients[client]['to_client'].put(Message())

            sleep(1)

server = Server()
client1 = Client(server.control, "a")
client2 = Client(server.control, "b")
server.start()
client1.start()
client2.start()
server.join()

Collectives™ on Stack Overflow

Python , multithreading with Queue

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related