initialize multiple numpy array efficiently

Question

I need to initialize multiple numpy arrays with the same shape. Wondering which way is the best to go:

write a line for each:

dist_x=np.zeros((1,len(pose07)-1))
dist_y=np.zeros((1,len(pose07)-1))
rots_absulute=np.zeros((1,len(pose07)-1))
rots=np.zeros((1,len(pose07)-1))

calculate the length first and save it as a parameter:

length=len(pose07)-1

dist_x=np.zeros(1,length)
dist_y=np.zeros(1,length)
rots_absulute=np.zeros(1,length)
rots=np.zeros(1,length)

initialize one of them, then copy it multiple times:

dist_x=np.zeros((1,len(pose07)-1))
dist_y=np.copy(dist_x)
rots_absulute=np.copy(dist_x)
rots=np.copy(dist_x)

or maybe there is a better way?

Haven't timed it, but I think we can reasonably guess that copying will be slower - no need to read in data known to be zeros, compared to just writing zeros. — alani
– alani, Commented Jul 4, 2020 at 13:06
Does this answer your question? stackoverflow.com/a/20860276/11667949 — Shivam Jha
– Shivam Jha, Commented Jul 4, 2020 at 13:20
Consider np.empty_like or np.zeros_like. At least it looks cleaner. If you know for sure that you're going to overwrite all elements, np.empty can be faster than np.zeros, depending on whether it's memory allocated via the operating system or re-used memory that was already allocated by the process. — Han-Kwang Nienhuys
– Han-Kwang Nienhuys, Commented Jul 4, 2020 at 14:51
It depends on your definition of "better". It's unlikely you mean speed because any halfway decent algorithm is likely to take many times longer to run than the initial allocation. If you mean readability, they could all be improved by adding a comment explaining what they are for. — Mark Setchell
– Mark Setchell, Commented Jul 4, 2020 at 16:06
Not that it affects speed, but why are you using (1,length) as opposed to just (length,)? — hpaulj
– hpaulj, Commented Jul 4, 2020 at 16:33

Hammad Ahmed · Accepted Answer · 2020-07-04 18:39:58Z

1

arr1, arr2, arr3, arr4 = np.zeros((4, 1, len(pose07)-1))

EDIT: ran some bench-marks

def func1(inp):
    
    a = np.zeros([1, len(inp)])
    b = np.zeros([1, len(inp)])
    c = np.zeros([1, len(inp)])
    d = np.zeros([1, len(inp)])
    
    return 1

def func2(inp):
    
    length = len(inp)

    a = np.zeros([1, length])
    b = np.zeros([1, length])
    c = np.zeros([1, length])
    d = np.zeros([1, length])
    
    return 1

def func3(inp):
    
    a = np.zeros([1, len(inp)])
    b = np.copy(a)
    c = np.copy(a)
    d = np.copy(a)
    
    return 1

def func4(inp):
    
    a, b, c, d = np.zeros([4, 1, len(inp)])
    
    return 1

import perfplot

bench = perfplot.bench(
    n_range= range(1, 50),
    setup= lambda n: np.random.randint(100, size= [n]),
    kernels= [
        func1,
        func2,
        func3,
        func4
    ]
)

bench.show()

according to the benchmarks func2 (where length is stored in a variable first) seems to perform the best,

and try to avoid func 4, as pointed out by Paul Panzer in the comment below, func 4 can cause memory leaks

edited Jul 4, 2020 at 18:39

answered Jul 4, 2020 at 15:50

Hammad Ahmed

8858 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Paul Panzer Over a year ago

Depending on how arr1,...,arr4 are going to be used, specifically, if one, say, lives much longer than the others, this may be a memory leak.

zheyuanWang Over a year ago

very reliable! Could you please run an extra experiment comparing the speed of "a, b, c = [np.zeros(5) for _ in range(3)]" according to stackoverflow.com/a/20860276/11667949 ?

pip · Accepted Answer · 2023-08-22 10:31:00Z

0

A nice concise way using multiple-assignment:

length=len(pose07)-1
dist_x, dist_y, rots_absolute, rots = 4 * [np.zeros(1, length)]

answered Aug 22, 2023 at 10:31

pip

2632 silver badges8 bronze badges

Collectives™ on Stack Overflow

initialize multiple numpy array efficiently

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related