This appears to be a simple issue, but I've been struggling trying to efficiently split a 2D array:
start_time = time.time()
M = np.ones((400,400))
for i in range(10000):
e = np.array_split(M, 20)
print time.time() - start_time
However, this process takes ~6 seconds comparing to ~0.5 seconds when implemented in Mathematica with the Partition function, which can become a liability when the array gets much larger. Is there any way for me to speed up the process?
np.array_split may be useful when splitting an array into uneven pieces. Here, the size of each item in e is the same, so you could just use reshape:
e = M.reshape(20,-1)
This will be exceedingly fast, since it requires no copying of the array, only a change to the array's shape attribute.
e will be a 2D NumPy array of shape (20, 8000), not a list of NumPy arrays.
In [56]: M = np.ones((400,400))
In [60]: %timeit M.reshape(20,-1)
1000000 loops, best of 3: 447 ns per loop
-1 in the call to reshape tells reshape to fill 'er up with whatever number is necessary to use all the items in M. Since M has shape (400, 400), there are 160000 items. If the first axis has length 20, then the second axis must have length 160000/20 = 8000. Instead of doing the math, you just put -1, and let reshape do the work.
single evaluation of e = np.array_split(M, 20) needs in reality 10000 times less.np.shape(e) is (20, 20, 400) and different from unutbus answer. He is measuring the mean time for only once calculating M.reshape(20,-1). I get%timeit Mr=M.reshape(20,-1) 562 ns ± 55.7 nsand for your case:%timeit e=np.array_split(M, 20) 93.6 µs ± 4.72 µs. This is a factor of about 200 slower, but the result is another array than in ubuntus answer, which givesnp.shape(Mr) (20, 8000).