Multithreading in Python within a for loop

Question

Let's say I have a program in Python which looks like this:

import time

def send_message_realtime(s):
    print("Real Time: ", s)
    
def send_message_delay(s):
    time.sleep(5)
    print("Delayed Message ", s)

for i in range(10):
    send_message_realtime(str(i))
    time.sleep(1)
    send_message_delay(str(i))

What I am trying to do here is some sort of multithreading, so that the contents of my main for loop continues to execute without having to wait for the delay caused by time.sleep(5) in the delayed function.

Ideally, the piece of code that I am working upon looks something like below. I get a message from some API endpoint which I want to send to a particular telegram channel in real-time(paid subscribers), but I also want to send it to another channel by delaying it exactly 10 minutes or 600 seconds since they're free members. The problem I am facing is, I want to keep sending the message in real-time to my paid subscribers and kind of create a new thread/process for the delayed message which runs independently of the main while loop.

def send_message_realtime(my_realtime_message):
    telegram.send(my_realtime_message)


def send_message_delayed(my_realtime_message):
    time.sleep(600)
    telegram.send(my_realtime_message)

while True:
    my_realtime_message = api.get()
    send_message_realtime(my_realtime_message)
    send_message_delayed(my_realtime_message)

dreamcrash · Accepted Answer · 2021-02-20 16:59:51Z

1

I think something like ThreadPoolExecutor does what you are look for:

import time
from concurrent.futures.thread import ThreadPoolExecutor


def send_message_realtime(s):
    print("Real Time: ", s)


def send_message_delay(s):
    time.sleep(5)
    print("Delayed Message ", s)


def work_to_do(i):
    send_message_realtime(str(i))
    time.sleep(1)
    send_message_delay(str(i))


with ThreadPoolExecutor(max_workers=4) as executor:
    for i in range(10):
        executor.submit(work_to_do, i)

The max_workers would be the number of parallel messages that could have at a given moment in time.

Instead of a multithread solution, you can also use a multiprocessing solution, for instance

from multiprocessing import Pool
...
with Pool(4) as p:
    print(p.map(work_to_do, range(10)))

edited Feb 20, 2021 at 16:59

answered Nov 11, 2020 at 18:53

dreamcrash

52.5k26 gold badges114 silver badges138 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Multithreading in Python within a for loop

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related