Configure python sqlite3 module to use different (newer version) sqlite system lib (Upgrade sqlite)

Question

We need to use some of the newest sqlite feature like "UPDATE FROM", which is available from sqlite 3.33.0 (https://www.sqlite.org/changes.html).

In current debian docker image we use stable (buster) distribution, which has older version of sqlite lib (the name is libsqlite3-dev).

I compiled sqlite3 locally from sources in .sh script, but I'm not able to configure pythons sqlite3 module to use the just compiled version of sqlite.

This is how the code looks like:

# Download newer libsqlite3-dev with "UPDATE FROM" clause support
cd data_processing
wget "https://www.sqlite.org/2020/sqlite-amalgamation-3340000.zip" -O sqlite.zip
unzip sqlite.zip
rm sqlite.zip
mv sqlite-amalgamation-3340000 sqlite-3.34
cd sqlite-3.34
gcc shell.c sqlite3.c -lpthread -ldl -o sqlite3
# remove no longer needed file (everything else than just created sqlite3)
find . ! -name 'sqlite3' -type f -exec rm -f {} +
cd -
export PATH="$(pwd)/my_app/sqlite-3.34:$PATH"

Here we can see, that adding the binary to the path is not enough. I also tried adding:

sys.path.insert(
    0,
    '/home/user/project/libs/my_app/sqlite-3.34'
)

into python code, no success. Is there any way to configure sqlite3 python module to use a different sqlite lib installed on the system?

Look at this question, and see if that might help you: stackoverflow.com/questions/14541869/… — JohanL
– JohanL, Commented Dec 4, 2020 at 19:10

Anthony · Accepted Answer · 2022-04-01 15:35:30Z

1

to add to answer above (not enough rep)

import sys
sys.modules['sqlite3'] = __import__('pysqlite3')

to swap in the binary in-place of system sqlite3

credit: https://romandc.com/zappa-django-guide/walk_database/

answered Apr 1, 2022 at 15:35

Anthony

1111 silver badge3 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

wbrycki · Accepted Answer · 2023-10-06 14:26:40Z

1

Use binary package of pysqlite3: https://github.com/coleifer/pysqlite3 in your requirements, add

pip install pysqlite3-binary

in python code, use pysqlite3:

import pysqlite3
    (...) conn = pysqlite3.connect(r"filename") (...)

instead of sqlite3.

Alternative, as Anthony suggested: swapping the system binary You can swap the binary of the system:

import sys
sys.modules['sqlite3'] = __import__('pysqlite3')

And then, you don't have to use pysqlite3 in your code.

edited Oct 6, 2023 at 14:26

answered Dec 8, 2020 at 12:24

wbrycki

1511 silver badge10 bronze badges

1 Comment

Cesar A. Mostacero Over a year ago

I'm getting not-found error while trying to install with pip: ERROR: No matching distribution found for pysqlite3-binary

Collectives™ on Stack Overflow

Configure python sqlite3 module to use different (newer version) sqlite system lib (Upgrade sqlite)

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related