2
$\begingroup$

I'm trying to run a Random Forest model from sklearn but I keep getting an error: ValueError: Input contains NaN, infinity or a value too large for dtype('float32').

I tried following steps in ValueError: Input contains NaN, infinity or a value too large for dtype('float32')

fillna(0) on my pandas dataframe still gave the ValueError.

So I tried working with my numpy array:

val = setTo.ravel().nan_to_num(0)

But I keep getting an error: 'numpy.ndarray' object has no attribute 'nan_to_num'

I'm wondering how I can deal with the nan values if I have ndarray?

Update

Thanks so much to @Beniamin H for all the help, as suggested, I rescaled the data, which I based on https://stackoverflow.com/questions/34771118/sklearn-random-forest-error-on-input and it worked!

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Hi and welcome to Data Science Stack Exchange :) $\endgroup$ Commented Jan 6, 2021 at 20:17

1 Answer 1

Reset to default
1
$\begingroup$

You are using the right method but in a wrong way :)

nan_to_num is a method of numpy module, not numpy.ndarray. So instead of calling nan_to_num on you data, call it on numpy module giving your data as a paramter:

import numpy as np
data = np.array([1,2,3,np.nan,np.nan,5])
data_without_nan = np.nan_to_num(data)

prints:

array([1., 2., 3., 0., 0., 5.])

In your example:

import numpy as np
val = np.nan_to_num(setTo.ravel())
Improve this answer
$\endgroup$
12
  • $\begingroup$ Thanks so much! I tried val = np.nan_to_num(setTo.ravel()) but I still seem to be getting: ValueError: Input contains NaN, infinity or a value too large for dtype('float32'). $\endgroup$ Commented Jan 6, 2021 at 22:22
  • $\begingroup$ I'm also not sure if it's due to a NaN value or possibly a value too large. I don't seem to see any NaN in the actual data $\endgroup$ Commented Jan 6, 2021 at 22:26
  • $\begingroup$ You may try to use pandas replace: df.replace([np.inf, -np.inf], 0) to replace inf and -inf with 0 $\endgroup$ Commented Jan 6, 2021 at 22:58
  • $\begingroup$ There are more answers which may help you: stackoverflow.com/questions/31323499/… $\endgroup$ Commented Jan 6, 2021 at 23:00
  • $\begingroup$ Thanks again for your help! So I tried df.replace([np.inf, -np.inf], 0) but still getting the error. I tried some of the solutions in stackoverflow.com/questions/31323499/…, np.isfinite(X) showed true so that must be the problem!! $\endgroup$ Commented Jan 6, 2021 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.