async and await with Python

Question

I have worked with async/await in C# before as in this question, but it is a concept that I need to go to the basics for a refresh from time to time.

This time I am working in Python, but this question is more about the concept of Async Await than the implementation.

Basically I have a server to which I send a request and this server sends a response. If it is only once I have

async def send_and_get_response(self,request):
   self.remote_server.send_request(request)
   response= await self.remote_server.get_response()
   return response

No problem there.

However, imagine that I want to send not one but several requests to the server, and I want to await for the responses. I don't want to send one request after I get the response, but all at once. I don't want to miss responses either.

Correct me if I am wrong but the nature of async/await is that if I put code after await this code won't get executed until the wait is complete right?

So this won't do right?

   async def send_and_get_response(self,request1,request2):
       self.remote_server.send_request(request1)
       response1= await self.remote_server.get_response()
       self.remote_server.send_request(request2)
       response2= await self.remote_server.get_response()
       return [response1,response2]

I imagine that in the above case, request2 will get send only after response1 has arrived, am I wrong?

So what to do? I thought something like

 async def send_and_get_response(self,request1,request2):
       self.remote_server.send_request(request1)
       self.remote_server.send_request(request2)
       response1= await self.remote_server.get_response()
       response2= await self.remote_server.get_response()
       return [response1,response2]

but this somehow does not feel right. What if response1 arrives immediately before I could await it? What if response2 arrives before than response1?

So how about this one:

async def send_and_get_response(self,request):
   self.remote_server.send_request(request)
   response= await self.remote_server.get_response()
   return response

response1=send_and_get_response(request1)
response2=send_and_get_response(request2)

Will this work?

I repeat, what I want to accomplish is to send two (or more) requests to the server and await the response from all the requests. They can come in a different order.

Edit

How about this one

    async def send_and_get_response(self,request):
       self.remote_server.send_request(request)
       response= await self.remote_server.get_response()
       return response
    
 results=  await asyncio.gather(
            send_and_get_response(request1),
            send_and_get_response(request2),
            send_and_get_response(request3),
        )

Will this work? If it does, how do I get the responses?

Edit 2

I have checked other resources and I wonder how to make the following work

async def process_when_done(tasks):
    for res in asyncio.as_completed(tasks):
        print("Result %s" % await res)    
    
tasks2= [ send_and_get_response(request1),
        send_and_get_response(request2),
        send_and_get_response(request3)]   
    
process_when_done(tasks2)

In this case I am trying to process every process as soon as it finishes

Sraw · Accepted Answer · 2020-05-25 01:40:31Z

1

I imagine that in the above case, request2 will get send only after response1 has arrived, am I wrong?

You are correct.

but this somehow does not feel right. What if response1 arrives immediately before I could await it? What if response2 arrives before than response1?

It could be right, as you want to wait for all the responses(Which means you are actually waiting for the longest response). It may not be right if you need to do something with the response immediately:

async def send_and_get_response(self,request1,request2):
    self.remote_server.send_request(request1)
    self.remote_server.send_request(request2)
    response1= await self.remote_server.get_response()
    # do something here with response1
    # if response2 arrives first, then we are wasting time.
    response2= await self.remote_server.get_response()
    # do something here with response2
    return [response1,response2]

will this work?

No, send_and_get_response is an async function, you need to await send_and_get_response, or you do nothing, but if you await it, then there is no difference from the first implementation.

Actually we use asyncio.gather for this purpose in Python.

answered May 25, 2020 at 1:40

Sraw

20.6k11 gold badges61 silver badges93 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

KansaiRobot Over a year ago

I see, so the last one will not work. I see it now. ;( I will check your link, but do you have some idea how to do what I wish to do?

Sraw Over a year ago

Your second implementation actually works. Or you can follow the link to use a more pythonic way.

KansaiRobot Over a year ago

I have added one more try with asyncio.gather. Could you take a look please?

Sraw Over a year ago

Yeah, it makes sense.

Sraw Over a year ago

"The order of result values corresponds to the order of awaitables in aws." See the document.

|

Collectives™ on Stack Overflow

async and await with Python

Edit

Edit 2

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Edit

Edit 2

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related