Replace null values with other values using Python conditional

Question

I have 2 dataframes with different number of row. So, lets say:

df1 = {'Name': ['Tom', 'Joseph', 'Krish', 'John','Micheal'], 'Age': [20, 21, 19, 18, 23], 'HeightEach': [156, 167,149 , 151, 149]}  
df2 = {'Name': ['Tom', 'Joseph', 'Krish', 'John'], 'Age': [20, 21, 19, 18], 'Height': [179, NaN,159 , NaN]}

The issue now is , I want to replace the null values in 'Height' column with the same value in df1. So far I only know how to replace NaN with 0 like below code:

 df1['Height'] = df1['Height'].replace(np.nan, 0)

However, i do not know how to replace the NaN with this condition:

Replace NaN with same height in df1 if the name in df1 and df2 is same. I tried to do like simple code below well at least to get the data with the same name:

df1['Height'] = df1["Name"] == df2["Name"]

However it returns this error:

Edit: dataset for both data is not same. So, I just want to save the NaN data and ignore the existing data in df2.

So, i've tried:

if df1["Name"] == df2["Name"]:
    result = df2.merge(df1, how='inner')
    result.loc[result['Height'].isna(), 'Height'] = result['HeightEach']

It throws error:

df1['Height'][df1['Height'] == np.nan] = df2['Height']. Might also need to df1.set_index('Name') — Alexey S. Larionov
– Alexey S. Larionov, Commented Jun 17, 2021 at 9:55

ThePyGuy · Accepted Answer · 2021-06-17 10:13:16Z

1

Left merge df2, and df1, then fill NaN values by HeightEach

result = df2.merge(df1, how='left')
result.loc[result['Height'].isna(), 'Height'] = result['HeightEach']

     Name  Age  Height  HeightEach
0     Tom   20   156.0         156
1  Joseph   21   167.0         167
2   Krish   19   149.0         149
3    John   18   151.0         151

It may not work in cases when there isn't value in df1 for all values in df2, you may want to do inner merge first and fill the NaN values in that scenario.

result = df2.merge(df1, how='inner')
result.loc[result['Height'].isna(), 'Height'] = result['HeightEach']

df2.append(result)[['Name', 'Age', 'Height']].drop_duplicates(keep='last').dropna()
     Name  Age  Height
0     Tom   20   156.0
1  Joseph   21   167.0
2   Krish   19   149.0
3    John   18   151.0

edited Jun 17, 2021 at 10:13

answered Jun 17, 2021 at 10:03

ThePyGuy

18.5k5 gold badges24 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Replace null values with other values using Python conditional

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related