0

I am using sqlite database to store the data. My python program has a thread pool which contains 5 threads. I was creating a single database connection and sharing it with all 5 threads but some times it throws exception which was not captured by any sqlite exception or generic exception and my python script got killed automatically. After searching for the solution I came across How to share single SQLite connection in multi-threaded Python application

And I created a separate connections as follows,

class ProcessJob(object):
    ....
    def process_job(self):
        job = queue.get()
        if job = 'xyz':
            with sqlite3.connect(database_path, check_same_thread=False, timeout = 10) as db_conn:
                db_conn.execute("insert query on table ABC")
                db_conn.commit()
        elif job = 'pqr':
            with sqlite3.connect(database_path, check_same_thread=False, timeout = 10) as db_conn:
                db_conn.execute("update query on table ABC")
                db_conn.commit()
        elif job = 'mno':
            with sqlite3.connect(database_path, check_same_thread=False, timeout = 10) as db_conn:
                db_conn.execute("insert query on table FOO")
                db_conn.commit()

class MyThread(therading.Thread):
    ....
    process_job_obj = ProcessJob()
    def run(self):
        while True: 
            try:
                process_job_obj.process_job()
            except Exception as e:
                logger.exception('Exception : %s'%e)

def main():
    for i in range(5):
        trd = MyThread()
        trd.start()
if __name__ == "__main__":
    main()

So, Is this a right approach or is there any flaw or chances of stopping/killing the python script?

4
  • Looks good to me. An improvement you could do is use sqlite3.connect(database_path, timeout = 10) only once (no need to open the database at each query) Commented Apr 11, 2015 at 8:40
  • @ValentinLorentz Can you please explain 'only one connection' thing with example, because I think if I create only one connection then multiple thread going to share that connection and which is not recommended. Commented Apr 11, 2015 at 9:04
  • Because there will be only one ProcessJob instance, so all queries will be done from the same thread. Commented Apr 11, 2015 at 9:06
  • Oh, wait, I got it. I thought ProcessJob was running in its own thread. Which would be, I think, the right solution. Commented Apr 11, 2015 at 9:07

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.