I was looking at the Python unit tests for sqlite3 today and I noticed something awesome
sqlite.connect(":memory:")
This is a really cool idea that makes unit tests much easier to write. But then I started wondering, does sqlite3 always dealloacte memory used? Is sqlite3 smart enough to deallocate in interactive mode?
OSX Activity monitor was not reporting correct memory usage. use htop to see correct usage statistics.
First I started the interpreter in interactive mode
python -i
Then in the interpreter I wrote
import sqlite3 as sqlite
db = sqlite.connect(":memory:")
db.execute("create table test(id integer primary key, test_text text)")
for i in range(50000000):
db.execute("insert into test(test_text) values (?)", ("test",))
The first screen shot is the memory usage while interactive python is running. The memory usage is ~488mb.
Small Note, OSX activity monitor identifies the python interpreter as "Terminal" probably because it doesn't check for children's memory usage (doesn't look at forks).
This second screenshot is after exit() is called in the interpreter. The memory usage is still at ~488mb In short, the memory isn't deallocated on interpreter exit.
Update: Original test was 3.4. Just confirmed with 2.7.
Update2:
Attempted the test again with htop.
Apparently the lesson isn't that python.sqlite3 doesn't deallocate in interactive mode, it's that you shouldn't use OSX activity monitor to test memory usage.
Nothing to see here, move along.
Yes.
Before:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
ryan 27162 0.4 0.2 29492 5348 pts/7 S+ 15:17 0:00 python
After inserting 500000 rows:
ryan 27162 14.3 0.7 45436 15064 pts/7 S+ 15:17 0:13 python
After db.close()
ryan 27162 11.9 0.4 38620 8368 pts/7 S+ 15:17 0:13 python
However, after inserting 500000 rows and then doing db.execute("drop table test"), memory usage was not freed again. Creating a new test db and inserting rows again did not cause additional memory allocations, so it looks like the memory just wasn't returned to the OS.
