More Numpy Vectorization instead of using nested loops

Question

I have all_data as a numpy array with the size of (2,601), NUM_SAMPLES = 601 and NUM_CLUSTERS = 3. Is there any vector form to build f (a (601,9) numpy array) than what using nested for-loops as follows?

f = np.empty((0,9), float)
for n in range(NUM_SAMPLES):
    f_n = np.array([[]])
    for m in range(NUM_CLUSTERS):
        f_n = np.hstack( (f_n , z_i(alldata[:,n], m).T))
    f = np.concatenate((f, f_n) , axis=0)

NOTE : when recalling function z_i(alldata[:,n], m), it returns a (3,1) numpy array.

f is supposed to be 'F' in the following formula: formula of f

why? What is wrong? Could you please tell me more? @QuangHoang — Fatemeh Sangin
– Fatemeh Sangin, Commented Nov 30, 2021 at 20:26
what Quang want to say is, that you could create a function that can input all data and return a 3xn array or given an array and a m creates a 3m xn or something like that — Ulises Bussi
– Ulises Bussi, Commented Nov 30, 2021 at 20:52
z_i has to be called once for each combination of n and m.? And it's a relatively slow python function? That's your bottleneck! — hpaulj
– hpaulj, Commented Nov 30, 2021 at 21:39

CJR · Accepted Answer · 2021-11-30 20:54:34Z

1

Because you have some function z_i in the middle of your loops, you're more or less stuck with loops. You don't need to do a bunch of really inefficient concats like you're doing, but your array size is so small it probably doesn't matter.

f = np.vstack((np.hstack((z_i(alldata[:,n], m).T for m in range(NUM_CLUSTERS)))
               for n in range(NUM_SAMPLES)))

If you really want this to run faster you have to look into z_i and change how that's working.

answered Nov 30, 2021 at 20:54

CJR

3,9872 gold badges13 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

More Numpy Vectorization instead of using nested loops

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related