How to initialise an integer array.array object with zeros in Python

Question

Questions with similar titles are about Python lists or NumPy. This is about the array.array class part of the standard Python library, see https://docs.python.org/2/library/array.html

The fasted approach I came up with (for integer types) is to use array.fromfile with /dev/zero. This is

about 27 times faster than array.array('L', [0] * size), which temporarily requires more than twice the memory than for the final array,
about 4.7 times faster than arrar.array('L', [0]) * size
and over 200 times faster than using a custom iterable object (to avoid creating a large temporary list).

However, /dev/zero may be unavailable on some platforms. Is there a better way to do this without NumPy, non-standard modules or my own c-extension?

Demonstrator code:

import array
import sys
import time

size = 100 * 1000**2
test = sys.argv[1]

class ZeroIterable:
    def __init__(self, size):
        self.size = size
        self.next_index = 0
    def next(self):
        if self.next_index == self.size:
            raise StopIteration
        self.next_index = self.next_index + 1
        return 0
    def __iter__(self):
        return self

t = time.time()
if test == 'Z':
    myarray = array.array('L')
    f = open('/dev/zero', 'rb')
    myarray.fromfile(f, size)
    f.close()
elif test == 'L':
    myarray = array.array('L', [0] * size)
elif test == 'S':
    myarray = array.array('L', [0]) * size
elif test == 'I':
    myarray = array.array('L', ZeroIterable(size))     
print time.time() - t

I found stackoverflow.com/questions/2214651/… but it is more about fast access (updating elements, in particular incrementing counters) than about initialisation. — Joachim Wagner
– Joachim Wagner, Commented May 21, 2016 at 12:14
I found stackoverflow.com/questions/3214288/… but the question specifically asks for a non-zero value. — Joachim Wagner
– Joachim Wagner, Commented May 21, 2016 at 12:17
I can't come up with a faster method either. For non-zero values, array('L', islice(repeat(value), size) is pretty good too. — Martijn Pieters
– Martijn Pieters, Commented Sep 15, 2018 at 21:13
I’m surprised this doesn’t have more upvotes. Still have this question 5 years after you asked it. I would have hoped the Array class documentation would address this. /dev/zero is a clever solution other than it won’t work on Windows of course. However it is worth noting that other question linked shows a benchmark that list actually performs well compute wise, but doesn’t address the memory difference. — Rob T.
– Rob T., Commented Feb 19, 2024 at 16:31

0xF · Accepted Answer · 2020-02-05 12:26:26Z

3

Updated to Python 3 and added the 'B' method:

import array
import sys
import time

size = 100 * 1000**2
test = sys.argv[1]

class ZeroIterable:
    def __init__(self, size):
        self.size = size
        self.next_index = 0
    def __next__(self):
        if self.next_index == self.size:
            raise StopIteration
        self.next_index = self.next_index + 1
        return 0
    def __iter__(self):
        return self

t = time.time()
if test == 'Z':
    myarray = array.array('L')
    f = open('/dev/zero', 'rb')
    myarray.fromfile(f, size)
    f.close()
elif test == 'L':
    myarray = array.array('L', [0] * size)
elif test == 'S':
    myarray = array.array('L', [0]) * size
elif test == 'I':
    myarray = array.array('L', ZeroIterable(size))
elif test == 'B':
    myarray = array.array('L', bytes(size * 8))
print(len(myarray))
print(time.time() - t)

The 'S' method (array.array('L', [0]) * size) wins:

$ python3 --version
Python 3.7.3
$ python3 z.py Z
100000000
1.1691830158233643
$ python3 z.py L
100000000
2.712920665740967
$ python3 z.py S
100000000
0.6910817623138428
$ python3 z.py B
100000000
0.9187061786651611
$ python3 z.py I
100000000
62.862160444259644

answered Feb 5, 2020 at 12:26

0xF

3,8781 gold badge30 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Joachim Wagner Over a year ago

On my system, B is slightly faster than S in both Python 3.6 and 3.9 (about 1.1%).

Collectives™ on Stack Overflow

How to initialise an integer array.array object with zeros in Python

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related