Stacking numpy arrays with padding

Question

I have a list of 32 numpy arrays, each of which has shape (n, 108, 108, 2), where n is different in each array. I want to stack all of them to create a numpy array of shape (32, m, 108, 108, 2), where m is the maximum among the ns, and the shorter arrays are padded with zeros.

How do I do this?

I asked something similar yesterday, but the answers there seem to break when using deep arrays like in my case.

Concretely, I went with this solution in the end, which produced the cleanest code:

data = np.column_stack(zip_longest(*data, fillvalue=0))

But now it is throwing this error:

ValueError: setting an array element with a sequence.

You got some good answers yesterday, and good examples of how to express and test the ideas. Please follow those examples, and show us how you attempted to solve the new problem. Dare I flag this with minimal reproducible example? — hpaulj
– hpaulj, Commented Oct 30, 2018 at 19:09
One quick thought - you could reshape your arrays to 2d, (n, 108*108*2), and apply one of the 2d solutions. — hpaulj
– hpaulj, Commented Oct 30, 2018 at 19:10
What do you mean by break? Exception is thrown, not the wanted output? — Dani Mesejo
– Dani Mesejo, Commented Oct 30, 2018 at 19:13
zip_longest will work, but requires the right fillvalue. Not an integer, but an array of the right shape (e.g. np.zeros((108,108,2))) — hpaulj
– hpaulj, Commented Oct 30, 2018 at 19:42

Jsevillamol · Accepted Answer · 2018-10-30 19:27:00Z

4

I have found a godly answer in this webpage.

The pad_sequences function is exactly what I needed.

from tensorflow.python.keras.preprocessing.sequence import pad_sequences
result = pad_sequences(imgs, padding='post')

answered Oct 30, 2018 at 19:27

Jsevillamol

2,5712 gold badges28 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

D.Shaulskii · Accepted Answer · 2019-02-12 14:55:29Z

In my case I needed to stack images with different width and padded with zeros to the left side. for me this works well:

np.random.seed(42)
image_batch = []
for i in np.random.randint(50,500,size=10):
image_batch.append(np.random.randn(32,i))
for im in image_batch:
    print(im.shape)

output: (32, 152) (32, 485) (32, 398) (32, 320) (32, 156) (32, 121) (32, 238) (32, 70) (32, 152) (32, 171)

def stack_images_rows_with_pad(list_of_images):
    func = lambda x: np.array(list(zip_longest(*x, fillvalue=0))) # applied row wise
    return np.array(list(map(func, zip(*list_of_images)))).transpose(2,0,1)

res = stack_images_rows_with_pad(image_batch)

for im in rez:
    print(im.shape)

output: (32, 485) (32, 485) (32, 485) (32, 485) (32, 485) (32, 485) (32, 485) (32, 485) (32, 485) (32, 485)

Rocky Li · Accepted Answer · 2018-10-30 19:17:11Z

2

Try this:

# Create matrices with random first axis length.
depth = np.random.randint(3,20,size=32)
l = []
lmax = 0
for i in depth:
    l.append(np.ones((i,10,10,2)))
    lmax = i if i > lmax else lmax

# Join the matrices:
new_l = []
for m in l:
    new_l.append(np.vstack([m, np.zeros((lmax-m.shape[0], 10, 10, 2))]))
master = np.stack(new_l, axis=0)
master.shape
>>> (32, 19, 10, 10, 2)

I find np.pad almost impossible to work with on higher dimensional matrix - luckily, what you asked was simple, where only one of the dimension will have to extended, such that it's easy to use np.vstack to stack a zeros array that make it conform to a new shape.

answered Oct 30, 2018 at 19:17

Rocky Li

5,9862 gold badges21 silver badges36 bronze badges

Comments

Fabich · Accepted Answer · 2020-11-12 13:18:58Z

0

A = np.ones((4,3))


border_top_bottom = np.zeros((A.shape[1])).reshape(1,A.shape[1])
print(np.vstack([border_top_bottom,A,border_top_bottom]))

temp = np.vstack([border_top_bottom,A,border_top_bottom])

border_right_left = np.zeros((temp.shape[0])).reshape(temp.shape[0],1)
print(np.hstack([np.hstack([border_right_left,temp,border_right_left])]))

edited Nov 12, 2020 at 13:18

Fabich

3,1354 gold badges35 silver badges48 bronze badges

answered Nov 12, 2020 at 11:04

madracoon

232 bronze badges

Collectives™ on Stack Overflow

Stacking numpy arrays with padding

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related