Creating multiple async requests

Question

I'm just starting to explore this new library called Requests-Html, which I've just dicovered through Corey Schafer tutorial, and the challenge is to create a async call of n different requests.

So, for example, we have the following code, which runs in about 3,6 seconds:

async def get_delay1():
    r = await asession.get("https://httpbin.org/delay/1")
    return r

async def get_delay2():
    r = await asession.get("https://httpbin.org/delay/2")
    return r

async def get_delay3():
    r = await asession.get("https://httpbin.org/delay/3")
    return r

asession = AsyncHTMLSession()

t1 = time.perf_counter()

results = asession.run(get_delay1, get_delay2, get_delay3)

for result in results:
    response = result.html.url
    print(response)

t2 = time.perf_counter()

print(t2 - t1)

The question is, what should I do if I want to create a 500 async request with this library? It can't be the case that I will have to code 500 different functions, right?

I've tried to create a list with the functions generators, so that I can automatically pass n different functions inside of it, :

tasks = [get_delay1, get_delay2, get_delay3]
results = asession.run(tasks)

But I get the

ERROR`: 
asyncio.ensure_future(coro()) for coro in coros
TypeError: 'list' object is not callable

Marco Antonio · Accepted Answer · 2020-01-15 23:10:13Z

1

I've found out how to do it.

After we create our list, we have to call

asession.run(*tasks)

This way, we ensure that we are passing only callables to the run method, i.e., elements of the list and not the list itself.

By!

edited Jan 15, 2020 at 23:10

answered Jan 15, 2020 at 4:05

Marco Antonio

1432 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Creating multiple async requests

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related