I was returning a dataframe of characters from GOT such that they were alive and predicted to die, but only if they have some house name. (important person). I was expecting it to skip NaN's, but it returned them as well. I've attached screenshot of output. Please help.
PS I haven't attached any spoilers so you may go ahead.
import pandas
df=pandas.read_csv('character-predictions.csv')
a=df[((df['actual']==1) & (df['pred']==0)) & (df['house'] !=None)]
b=a[['name', 'house']]
You need notnull with ix for selecting columns:
b = df.ix[((df['actual']==1) & (df['pred']==0)) & (df['house'].notnull()), ['name', 'house']]
Sample:
df = pd.DataFrame({'house':[None,'a','b'],
'pred':[0,0,5],
'actual':[1,1,5],
'name':['J','B','C']})
print (df)
actual house name pred
0 1 None J 0
1 1 a B 0
2 5 b C 5
b = df.ix[((df['actual']==1) & (df['pred']==0)) & (df['house'].notnull()), ['name', 'house']]
print (b)
name house
1 B a
You can also check pandas documentation:
Warning
One has to be mindful that in python (and numpy), the nan's don’t compare equal, but None's do. Note that Pandas/numpy uses the fact that np.nan != np.nan, and treats None like np.nan.
In [11]: None == None
Out[11]: True
In [12]: np.nan == np.nan
Out[12]: False
So as compared to above, a scalar equality comparison versus a None/np.nan doesn’t provide useful information.
In [13]: df2['one'] == np.nan
Out[13]:
a False
b False
c False
d False
e False
f False
g False
h False
Name: one, dtype: bool
np.nanin comparisons.np.nan == np.nanreturnsFalse. Furthermore, you are trying to comparenp.nan != None. Even a naive interpretation would tell you that is True. @jezrael's answer below gets you what you need.