Python3 numpy array size compare to list

Question

I always thought numpy array is more compact and takes less memory size compare to list, however, for a 3-D float64 np array,

print (sys.getsizeof(result2)/1024./1024./1024.)
print (sys.getsizeof(result2.astype('float16'))/1024./1024./1024.)
print (sys.getsizeof(list(result2))/1024./1024./1024.)
print (sys.getsizeof(result2.tolist())/1024./1024./1024.)

The output is,

0.6521792411804199
0.16304489970207214
0.00033977627754211426
0.0003019943833351135

The list took much smaller memory. Is it validate to use sys.getsizeof for list? If it is, can I do anything to improve np array memory use?

######################

Using pympler @J_H, (it seems pympler can't deal with arraies in a list, like list(a 3-D array ). )

print (result2.nbytes/1024./1024./1024.)
print (asizeof.asizeof(result2)/1024./1024./1024.)
print (result2.astype('float16').nbytes/1024./1024./1024.)
print (asizeof.asizeof(list(result2))/1024./1024./1024.)
print (asizeof.asizeof(result2.tolist())/1024./1024./1024.)

0.6521791219711304
0.6521792411804199
0.1630447804927826
0.004566863179206848
4.078836984932423

Thank you all!

getsizeof is non-recursive, i.e. it will not account for the conent of a container. — Paul Panzer
– Paul Panzer, Commented Jul 26, 2020 at 1:16
The memory use of a ndarray` is easy - arr.nbytes, or product of shape and bytes per element. Memory use of a list is hard to estimate. — hpaulj
– hpaulj, Commented Jul 26, 2020 at 1:21

J_H · Accepted Answer · 2020-07-26 01:35:41Z

1

You show that each of your list elements consumes 8 bytes.

But each element is just a pointer to a 24-byte float object.

Additionally, when you start with a 3-D array, you'll be looking at lists within lists. You could recurse through the data structures yourself to accurately add up the allocated bytes. Or you could use pympler.

answered Jul 26, 2020 at 1:35

J_H

21.2k5 gold badges29 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python3 numpy array size compare to list

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related