Creating an array without certain ranges

Question

In python I have numpy.ndarray called a and a list of indices called b. I want to get a list of all the values of a which are not in -10..10 places around the indices of b. This is my current code, which takes a lot of time to run due to allocations of data (a is very big):

    aa=a
    # Remove all ranges backwards
    for bb in b[::-1]:
        aa=np.delete(aa, range(bb-10,bb+10))

Is there a way to do it more efficiently? Preferably with few memory allocations.

Remember ranges do not include the "to" value, so your code will delete indexes bb-10,bb-9,...,bb+9. Is this what you intended? — Lauritz V. Thaulow
– Lauritz V. Thaulow, Commented Mar 3, 2012 at 12:42

Paul · Accepted Answer · 2012-03-04 00:17:08Z

2

np.delete will take an array of indicies of any size. You can simply populate your entire array of indicies and perform the delete once, therefore only deallocating and reallocating once. (not tested. possible typos.)

bb = np.empty((b.size, 21), dtype=int)
for i,v in enumerate(b):
    bb[i] = v+np.arange(-10,11)

np.delete(a, bb.flat)  # looks like .flat is optional

Note, if your ranges overlap, you'll get a difference between this and your algorithm: where yours will remove more items than those originally 10 indices away.

edited Mar 4, 2012 at 0:17

answered Mar 3, 2012 at 16:58

Paul

43.9k17 gold badges112 silver badges126 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Uri Cohen Over a year ago

Code execution time reduced from 1100 seconds to 9. :)

Lauritz V. Thaulow · Accepted Answer · 2012-03-03 13:02:58Z

0

Could you find a certain number that you're sure will not be in a, and then set all indices around the b indices to that number, so that you can remove it afterwards?

import numpy as np
for i in range(-10, 11):
    a[b + i] = number_not_in_a
values = set(np.unique(a)) - set([number_not_in_a])

This code will not allocate new memory for a at all, needs only one range object created, and does the job in exactly 22 c-optimized numpy operations (well, 43 if you count the b + i operations), plus the cost of turning the unique return array into a set.

Beware, if b includes indices which are less than 10, the number_not_in_a "zone" around these indices will wrap around to the other end of the array. If b includes indices larger than len(a) - 11, the operation will fail with an IndexError at some point.

edited Mar 3, 2012 at 13:02

answered Mar 3, 2012 at 12:49

Lauritz V. Thaulow

51.3k13 gold badges76 silver badges94 bronze badges

2 Comments

Uri Cohen Over a year ago

So how do I get a copy of a without the number_not_in_a values?

Lauritz V. Thaulow Over a year ago

@Uri np.delete(np.where(a == number_not_in_a)) should do the trick. It seems to me Paul has the better idea though.

Collectives™ on Stack Overflow

Creating an array without certain ranges

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related