Get smallest N values from numpy array ignoring inf and nan

Question

I need a good, quick method for finding the 10 smallest real values from a numpy array that could have arbitrarily many nan and/or inf values.

I need to identify the indices of these smallest real values, not the values themselves.

I have found the argmin and nanargmin functions from numpy. They aren't really getting the job done because I also want to specify more than 1 value, like I want the smallest 100 values, for example. Also they both return -inf values as being the smallest value when it is present in the array.

heapq.nsmallest kind of works, but it also returns nan and -inf values as smallest values. Also it doesn't give me the indices that I am looking for.

Any help here would be greatly appreciated.

iterate over/copy the array, convert all nans and -inf into inf run your function to get smallest N values, convert them back/revert to the old copy? silly hacky, but hmm... — Patashu
– Patashu, Commented Apr 24, 2013 at 13:27
thanks for the help, that is what I will have to do if I can't get a simpler answer. — jeffery_the_wind
– jeffery_the_wind, Commented Apr 24, 2013 at 13:29

YXD · Accepted Answer · 2013-04-24 13:42:25Z

11

The only values that should be throwing this out are the negative infinite ones. So try:

import numpy as np
a = np.random.rand(20)
a[4] = -np.inf
k = 10
a[np.isneginf(a)] = inf
result = a[np.argsort(a)[:k]]

edited Apr 24, 2013 at 13:42

answered Apr 24, 2013 at 13:35

YXD

32.6k15 gold badges79 silver badges117 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

interjay Over a year ago

2*np.max doesn't work if all elements are negative, better to just use inf I think.

Hooked Over a year ago

The solution as it stands can be dangerous. It should be noted here that you are changing the array to make a measurement. It may be physically significant that -inf is not inf.

Blckknght Over a year ago

You can avoid modifying the original array if you want. Just sort unconditionally, then limit your results to finite values: i = np.argsort(a); result = i[np.isfinite(a[i])][:10]

YXD Over a year ago

@Blckknght yes that's a better solution. If you post that I'll remove my answer

askewchan · Accepted Answer · 2013-04-25 01:25:36Z

2

It seems to me like you could just take the first n finite values from your sorted array, instead of trying to modify the original array, which could be dangerous.

n = 10
b = np.sort(a)
smalls = b[np.isfinite(b)][n:]

answered Apr 25, 2013 at 1:25

askewchan

46.7k18 gold badges125 silver badges135 bronze badges

Comments

Moj · Accepted Answer · 2013-04-24 13:41:20Z

1

you can find the index of inf and Nan like this:

a=np.array([[12,12,111],[np.inf,np.inf,1,2,3],[np.nan,7,8]])

the you can loop through a and check it with:

for item in a:    
    idxInf=(np.isnan(a[item])).nonzero()
    idxNan=(np.isnan(a[item])).nonzero()

i.e:

In [17]: (np.isnan(a[2]))
Out[17]: array([ True, False, False], dtype=bool)

In [18]: (np.isnan(a[2])).nonzero()
Out[18]: (array([0]),)

answered Apr 24, 2013 at 13:41

Moj

6,3812 gold badges26 silver badges36 bronze badges

Collectives™ on Stack Overflow

Get smallest N values from numpy array ignoring inf and nan

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related