I have a numpy array with 3 dimensions. I want to iterate over 2 dims and pull everything out in the 3rd dim. I.E:
arr = numpy.random.rand(3,5,5)
for i in range(arr.shape[1]):
for j in range(arr.shape[2]):
print arr[:, i, j]
Is this the most efficient way to loop? I know numpy provides the more efficient nditer function for looping, but it doesn't seem like it is able to do stuff like this
The actual arrays I'll be using will have a size of about 30x256x256
You can use numpy.transpose for this:
>>> x, y, z = arr.shape
>>> np.transpose(arr, (1, 2, 0)).reshape(y*z, x)
array([[ 8.89189379e-01, 5.95637587e-01, 7.84594074e-01],
[ 4.46214496e-01, 6.95533725e-03, 5.99493854e-02],
[ 4.37458356e-01, 4.17801277e-01, 8.70384164e-01],
[ 1.22083367e-01, 3.15002894e-01, 9.61295653e-01],
[ 2.15219210e-01, 5.99682222e-01, 8.59042071e-01],
[ 7.39714387e-01, 6.06449305e-01, 1.53375491e-01],
[ 4.34580313e-01, 8.23793966e-01, 2.58262432e-01],
[ 6.53256475e-01, 9.10842288e-01, 6.62668876e-01],
[ 2.60638435e-01, 2.44083731e-01, 9.44411275e-01],
[ 3.46072029e-01, 3.36690811e-01, 5.56281161e-04],
[ 5.54365956e-01, 7.84576199e-01, 2.92020128e-01],
[ 6.98475648e-01, 7.59483427e-01, 8.09173748e-01],
[ 7.28369542e-01, 2.07783197e-01, 3.36918305e-01],
[ 3.64955373e-01, 2.09863710e-01, 4.68231831e-02],
[ 9.10347730e-01, 2.59136721e-01, 7.71923984e-01],
[ 6.86310347e-01, 5.99903493e-01, 1.93947009e-01],
[ 1.28353564e-01, 4.04525015e-01, 8.46140174e-01],
[ 4.54025659e-01, 8.81360670e-01, 4.43411994e-01],
[ 6.57856096e-01, 3.55154332e-02, 6.74960684e-01],
[ 8.58154335e-01, 2.44856092e-01, 7.33027949e-01],
[ 2.09503288e-01, 1.20565562e-01, 5.44488104e-01],
[ 4.67728847e-02, 6.54273408e-02, 4.70930711e-02],
[ 3.70647262e-02, 5.72090215e-01, 4.38541549e-01],
[ 7.30252318e-01, 4.96902990e-02, 5.80768124e-01],
[ 4.92665142e-01, 9.16531057e-01, 8.29183892e-01]])
Adapting my answer from recent question, you can use ndindex
https://stackoverflow.com/a/29467367/901925
for tup in np.ndindex((arr.shape[1:])):
tup1=(slice(None),tup[0],tup[1])
print arr[tup1]
This uses nditer to generate a multi_index, which can be combined with a slice to produce the desired index.
The nditer tutorial page also shows how a mix of order and external_loop properties can make an nditer return a sub_vector, but that is tricky.
nditer isn't more 'efficient' or faster. You still end up indexing each element, or in your case a slice on the 1st dimension. nditer is most useful as a step toward coding the problem cython. In pure Python it is just as slow, even slower, than for loops.
nditer is also great when you need to step through several arrays in unison, e.g. c[i] = a[i]+b[i].
If you must iterate over the last 2 dimensions, what you are doing is probably as fast as any other method. Other methods just hide details.
You might explore swapping axes, or reshaping, e.g. arr.reshape(3,-1).
for x in arr.reshape(3,-1).T:
print(x)
This is my winner in speed.
ndindex, is it possible to make it say give me the indexes which would give me all the 2nd layers of all the columns of all the rows? For example a shape of (3, 3, 3), I want to just iterate over (0,0,1), (0,1,1), (0,2,1) ....
ndindex for that, just do a 1d iteration: for i in range(3): idx=(0, i, 1)