Non-blocking SQL execution using Python

Question

My question might not have a clear answer so please let me know if what I am trying to do is unrealistic.

I have a Python script that runs several independent SQL statements. Due to timeout limits, waiting for the statements to finish execution is not an option. The statements are for maintenance and no output is expected. Is there a way to asynchronously trigger these?

For example, using psycopg2.cursor, I am expecting to do the following:

cursor.execute(sql_statement)
# Run next code block

Possible duplicate of How to use python to query database in parallel — GMB
– GMB, Commented Dec 8, 2018 at 1:12
Run them in threads as rd_nielsen suggests, just be conscious of blocking on locks on tables / rows, which could cause your parallel implementation to single thread — seayak
– seayak, Commented Dec 8, 2018 at 1:47

reartnew · Accepted Answer · 2018-12-08 10:45:13Z

0

The recipe is threading. Use it in this way:

from threading import Thread
Thread(target=cursor.execute, args=(sql_statement,)).start()

It's worth noting that your program can not exit properly untill these threads are finished. If this behaviour is improper for you, you may pay attention to the subprocess module, which is capable of creating independently-running tasks.

answered Dec 8, 2018 at 10:45

reartnew

2771 silver badge9 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

doug65536 Over a year ago

Where is the join call on the Thread? You can't the way you have it chained off the return value - it's lost.

sergzach Over a year ago

The question is about how to continue execution of a program waiting a query result in a handler - how I do understand non-blocking queries.

Collectives™ on Stack Overflow

Non-blocking SQL execution using Python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related