2

Given an array

d = np.random.randn(100)

and an index array

i = np.random.random_integers(low=3, high=d.size - 5, size=20)

how can I efficiently create a 2d array r with

r.shape = (20, 8)

such that for all j=0..19,

r[j] = d[i[j]-3:i[j]+5]

In my case, the arrays are quite large (~200000 instead of 100 and 20), so something quick would be useful.

3
  • Does low and high make any difference? Like low=0, high=d.size - 8 and d[i[j]:i[j]+8]? Commented Mar 20, 2013 at 16:04
  • yes, it does make a difference. if an element of i is <3, then i[j]-3 is negative. similar for the upper bound. Commented Mar 20, 2013 at 16:13
  • But if all(0<=elem<=92 for elem in i) is True then d[i[j]:i[j]+8] would be the same, right? Commented Mar 20, 2013 at 16:21

1 Answer 1

1

You can create a windowed view of your data, i.e. a (93, 8) array, where item [i, j] is item [i+j] of your original array, as:

>>> from numpy.lib.stride_tricks import as_strided
>>> wd = as_strided(d, shape=(len(d)-8+1, 8), strides=d.strides*2)

You can now extract your desired slices as:

>>> r = wd[i-3]

Note that wd is simply a view of your original data, so it takes no extra memory. The moment you extract r with arbitrary indices, the data is copied. So depending on how you want to use your r array, you may want to delay that as much as possible, or maybe even avoid it altogether: you can always access what would be row r[j] as wd[j-3] without triggering a copy.

Sign up to request clarification or add additional context in comments.

3 Comments

Don't use take here, unless you first rewrite the function. Its great to know that take is faster often, but it is at least generally a very bad idea (and certainly not faster) here.
@seberg I am guessing it is the copying, that has to happen no matter what, that makes it a bad idea, right? Will edit my answer: thanks!
Well, the normal slicing won't do the temporary copy I believe... So if you take only a few items, you could be bloating memory up big time...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.