How can I make my Python program use 4 bytes for an int instead of 24 bytes?

Question

To save memory, I want to use less bytes (4) for each int I have instead of 24.

I looked at structs, but I don't really understand how to use them. https://docs.python.org/3/library/struct.html

When I do the following:

myInt = struct.pack('I', anInt)

sys.getsizeof(myInt) doesn't return 4 like I expected.

Is there something that I am doing wrong? Is there another way for Python to save memory for each variable?

ADDED: I have 750,000,000 integers in an array that I wish to be able to use given an index.

I think you are trying to solve the problem of a solution to a problem. Could you elaborate more about the real problem? — Shiplu Mokaddim
– Shiplu Mokaddim, Commented Jun 25, 2019 at 21:35
In modern computers, saving just a few bytes is far more programmer work than is helpful. This is true even for most embedded computers or small ones like the Raspberry Pi. What you ask would only help if you wanted to do it for many integers held in a larger data structure, such as an array. Is that the case for you? Which data structure are you using? (The particular structure matters for your question.) — Rory Daulton
– Rory Daulton, Commented Jun 25, 2019 at 21:38
I have 750,000,000 integers in an array that I wish to be able to use given an index. — Hyrial
– Hyrial, Commented Jun 25, 2019 at 21:39
Do you want these 750M ints in memory to search? You can read it page by page from the original data source and search on it. — Shiplu Mokaddim
– Shiplu Mokaddim, Commented Jun 25, 2019 at 21:40

Rory Daulton · Accepted Answer · 2019-06-25 21:52:22Z

3

If you want to hold many integers in an array, use a numpy ndarray. Numpy is a very popular third-party package that handles arrays more compactly than Python alone does. Numpy is not in the standard library so that it could be updated more frequently than Python itself is updated--it was considered to be added to the standard library. Numpy is one of the reasons Python has become so popular for Data Science and for other scientific uses.

Numpy's np.int32 type uses four bytes for an integer. Declare your array full of zeros with

import numpy as np
myarray = np.zeros((750000000,), dtype=np.int32)

Or if you just want the array and do not want to spend any time initializing the values,

myarray = np.empty((750000000,), dtype=np.int32)

You then fill and use the array as you like. There is some Python overhead for the complete array, so the array's size will be slightly larger than 4 * 750000000, but the size will be close.

edited Jun 25, 2019 at 21:52

answered Jun 25, 2019 at 21:46

Rory Daulton

22.7k7 gold badges46 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Noctis Skytower Over a year ago

What are your thoughts on the array module that is included in the standard library?

Rory Daulton Over a year ago

@NoctisSkytower: As far as I can tell, the only advantage of the array module is that it is part of the standard library. It is inferior to numpy in just about every other way. I wouldn't be surprised if it remains only for compatibility reasons. As I wrote, including numpy in the standard library was seriously considered but rejected for upgrading reasons. One key fact is that other popular third-party libraries, including matplotlib and pandas, use numpy rather than array. I cannot think of any such library that uses array.

Collectives™ on Stack Overflow

How can I make my Python program use 4 bytes for an int instead of 24 bytes?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related