Concatenate numpy array within a for loop

Question

I am creating inside a for loop in each iteration of it a numpy array of size 20x30x30x3. I want to concatenate all of those numpy arrays into a bigger one. If the iteration steps are 100 then the numpy array I want should be2000x30x30x3. I tried to do with lists:

new_one_arr1_list = []
new_one_arr2_list = []
all_arr1 = np.array([])
for item in one_arr1: # 100 iterations
    item = np.reshape(item, (1, 30, 30, 3))
    new_one_arr1 = np.repeat(item, 20, axis=0)

    all_arr1 = np.concatenate(([all_arr1 , new_one_arr1 ]))
    ind = np.random.randint(one_arr2.shape[0], size=(20,))
    new_one_arr2= one_arr1[ind]

    new_one_arr1_list.append(new_one_arr1)
    new_one_arr2_list.append(new_one_arr2)

In each iteration step new_one_arr1 and new_one_arr2 they have size 20x30x30x3. In the end when I am converting new_one_arr1_list and new_one_arr2_list and the size it is 100x20x30x30x3. How can I have 2000x30x30x3 in the end in a numpy array?

EDIT: I tried to use concatenate to add the arrays within a numpy array all_arr1 using: all_arr1= np.concatenate(([all_arr1, new_one_arr1])) however, I received the message:

ValueError: all the input arrays must have same number of dimensions

you use // for comment instead of #. This code can't compile. — DiCaprio
– DiCaprio, Commented Oct 17, 2018 at 11:11
Appending to a list in a loop is easier, and faster, than repeated concatenate., — hpaulj
– hpaulj, Commented Oct 17, 2018 at 12:35

DiCaprio · Accepted Answer · 2018-10-17 11:26:44Z

2

In order to create the concatenation and work around the error, I initialized the array with None and tested if it is None in the loop. Thereby you do not have to worry about not fitting dimensions. However, i created some arrays for the ones you did only describe and ended up with a final dimesion of (400, 30, 30, 3). This fits in here, since 20*20 = 400. Hope this helps for you solution.

new_one_arr1_list = []
new_one_arr2_list = []
one_arr1 = np.ones((20,30,30,3))
one_arr2 = np.ones((20,30,30,3))
all_arr1 = None
count = 0
for item in one_arr1: # 100 iterations
    item = np.reshape(item, (1, 30, 30, 3))
    new_one_arr1 = np.repeat(item, 20, axis=0)

    # print(all_arr1.shape, new_one_arr1.shape)
    if all_arr1 is None:
        all_arr1 = new_one_arr1
    else:
        all_arr1 = np.concatenate(([all_arr1 , new_one_arr1 ]), axis=0)
    ind = np.random.randint(one_arr2.shape[0], size=(20,))
    new_one_arr2= one_arr1[ind]

    new_one_arr1_list.append(new_one_arr1)
    new_one_arr2_list.append(new_one_arr2)
    count += 1
print(count)
all_arr1.shape

answered Oct 17, 2018 at 11:26

DiCaprio

8331 gold badge7 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

DiCaprio Over a year ago

I guess so. Depends on your goals mate

Sheikh Arbaz · Accepted Answer · 2018-10-17 11:37:04Z

2

Use np.concatenate operation given in the documenation: https://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.concatenate.html

Don't concatenate in the first iteration, it'll raise dimension error, just copy it during the first iter. For the remaining iterations keep concatenating.

new_one_arr1_list = []
new_one_arr2_list = []
all_arr1 = np.array([])
firstIteration = True
for item in one_arr1: # 100 iterations


    item = np.reshape(item, (1, 30, 30, 3))
    new_one_arr1 = np.repeat(item, 20, axis=0)

    if firstIteration:
        all_arr1 = new_one_arr1
        firstIteration=False
    else:
        all_arr1 = np.concatenate(([all_arr1 , new_one_arr1 ]))
    ind = np.random.randint(one_arr2.shape[0], size=(20,))
    new_one_arr2= one_arr1[ind]

    new_one_arr1_list.append(new_one_arr1)
    new_one_arr2_list.append(new_one_arr2)

edited Oct 17, 2018 at 11:37

answered Oct 17, 2018 at 10:20

Sheikh Arbaz

3422 silver badges10 bronze badges

5 Comments

Jose Ramon Over a year ago

How can I use it inside the for-loop? I need for example to add an empty array in the first loop.

Jose Ramon Over a year ago

I tried to do something like all_arr1= np.array([]) before the for loop and then within the for loop all_arr1= np.concatenate(([all_arr1, new_one_arr1])), however, i am getting error all the input arrays must have same number of dimensions .

Sheikh Arbaz Over a year ago

initialize all_arr1 before the for loop. I mean run an iteration first separately then iterate for remaining 99.

Jose Ramon Over a year ago

Sorry I did not get the point. You mean to concatenate the numpy arrays within the list?

Sheikh Arbaz Over a year ago

In the first iteration don't concatenate. Do the below. all_arr1 = new_one_arr1. In the remaining iterations keep concatenating.

Collectives™ on Stack Overflow

Concatenate numpy array within a for loop

2 Answers 2

1 Comment

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related