Pandas Dataframe replace values in a Series

Question

I am trying to update my_df based on conditional selection as in:

my_df[my_df['group'] == 'A']['rank'].fillna('A+')

However, this is not persistence ... e.g: the my_df still have NaN or NaT ... and I am not sure how to do this in_place. Please advise on how to persist the the update to my_df.

From the answers below, you have two problems. First, is that you are not re-assign your results back to the dataframe. And, two you have index chaining going on. Index chaining in this situation can be avoid by using loc. — Scott Boston
– Scott Boston, Commented Aug 15, 2018 at 14:32

jezrael · Accepted Answer · 2018-08-15 14:31:19Z

2

Create boolean mask and assign to filtered column rank:

my_df = pd.DataFrame({'group':list('AAAABC'),
                     'rank':['a','b',np.nan, np.nan, 'c',np.nan],
                     'C':[7,8,9,4,2,3]})

print (my_df)
  group rank  C
0     A    a  7
1     A    b  8
2     A  NaN  9
3     A  NaN  4
4     B    c  2
5     C  NaN  3

m = my_df['group'] == 'A'
my_df.loc[m, 'rank'] = my_df.loc[m, 'rank'].fillna('A+')

print(my_df)
  group rank  C
0     A    a  7
1     A    b  8
2     A   A+  9
3     A   A+  4
4     B    c  2
5     C  NaN  3

edited Aug 15, 2018 at 14:31

answered Aug 15, 2018 at 14:26

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

BENY Over a year ago

Hi @jscriptor, I recommend accept this answer , since the clean code is equally important with logic . (Always using mask for the condition is good for future modify )

jscriptor Over a year ago

Thanks a lot. All answers worked but this breaking up into 2 ops help me see what I was doing wrong.

BENY · Accepted Answer · 2018-08-15 14:26:17Z

2

You need to assign it back

my_df.loc[my_df['group'] == 'A','rank']=my_df.loc[my_df['group'] == 'A','rank'].fillna('A+')

answered Aug 15, 2018 at 14:26

BENY

324k22 gold badges176 silver badges250 bronze badges

Comments

jpp · Accepted Answer · 2018-08-15 16:09:50Z

1

Your operations are not in-place, so you need to assign back to a variable. In addition, chained indexing is not recommended.

One option is pd.Series.mask with a Boolean series:

# data from @jezrael

df['rank'].mask((df['group'] == 'A') & df['rank'].isnull(), 'A+', inplace=True)

print(df)

   C group rank
0  7     A    a
1  8     A    b
2  9     A   A+
3  4     A   A+
4  2     B    c
5  3     C  NaN

answered Aug 15, 2018 at 16:09

jpp

166k37 gold badges301 silver badges362 bronze badges

Collectives™ on Stack Overflow

Pandas Dataframe replace values in a Series

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related