Proper way to "append" multi-dimensional numpy arrays?

Question

I have a 4d array x, for which I want to loop through the first axis, modify that 3d array, and add this modified array to a new 4d array y.

I am currently doing something like:

xmod = modify(x[0, :, :, :])
y = xmod.reshape(1, x.shape[1], x.shape[2], x.shape[3])
for i in range(1, x.shape[0]):
    xmod = modify(x[i, :, :, :])
    y = np.vstack((y, xmod))

I am guessing there is amuch cleaner to do this. How?

Python-level loops, and especially appending in a loop, are spectacularly slow ways to use NumPy. Can you make modify operate in a vectorized manner on all of x? — user2357112
– user2357112, Commented Nov 19, 2015 at 22:50
I would love to, but I don't know if I can. My modify is really the resizing of an image: xmod = scipy.misc.imresize(x[i,:,:,0], 1.3). I don't think imresize can be vectorized, but maybe I'm wrong? — Fequish
– Fequish, Commented Nov 19, 2015 at 22:54
Your best bet is probably to create a blank array of the appropriate final size and then slice-assign the results into it one by one. — user2357112
– user2357112, Commented Nov 19, 2015 at 23:00

hpaulj · Accepted Answer · 2015-11-19 23:21:41Z

2

If you must act on x one submatrix at a time you could do:

y = np.zeros_like(x)
for i in range(x.shape[0]):
    y[i,...] = modify(x[i,...])

e.g.

In [595]: x=np.arange(24).reshape(4,3,2)
In [596]: y=np.zeros_like(x)
In [597]: for i in range(x.shape[0]):
   .....:     y[i,...]=x[i,...]*2
   .....:     
In [598]: y
Out[598]: 
array([[[ 0,  2],
        [ 4,  6],
        ...
        [40, 42],
        [44, 46]]])

appending to lists is generally better than repeatedly 'appending' to arrays:

In [599]: y=[]
In [600]: for row in x:
   .....:     y.append(row*2)
   .....: 
In [601]: y=np.array(y)

for very large cases you could see if vstack (or concatenate axis=0) is faster. But you have to explicitly add a beginning dimension to the arrays.

In [615]: y=[]
In [616]: for row in x:
    y.append((row*2)[None,:])
   .....:     
In [617]: np.vstack(y)

edited Nov 19, 2015 at 23:21

answered Nov 19, 2015 at 23:08

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

MaxNoe Over a year ago

for subarray in x and np.empty_like would be faster.

hpaulj Over a year ago

In my first example I'd have to use for i,row in enumerate(x):, or may zip(x,y). In a situation like this I doubt if empty_like is that much faster than zeros_like. The iteration takes much more time.

Collectives™ on Stack Overflow

Proper way to "append" multi-dimensional numpy arrays?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related