How to avoid duplicate entries while joining two dataframes in Python?

Question

I have two dataframes wi,

Site_ID Town
1235    Fitzroy
2344    Glen Iris

another with site_id and business name.

Site_ID Business Name
1235    BAC
2344    RFG

I would like to have only matched records upon joining two df's like this. After performing below merge function,

merge_df_rf1 = pd.merge(df1.drop_duplicates(), df2, on='site_id' ,how='inner')

I am getting this output.

Site_ID Business Name   Town
1235    BAC            Fitzroy
1235    BAC            Fitzroy
2354    RFG            Glen Iris
2354    RFG            Glen Iris

Not sure where I am going wrong with my join statement.

Any help on this will be highly appreciated.

Thank you in advance for the support!

I cannot reproduce this behaviour with the provided DataFrames. Your code does not produce duplicates on my end. — Henry Ecker
– Henry Ecker ♦, Commented Aug 27, 2021 at 3:01

U13-Forward · Accepted Answer · 2021-09-01 06:15:14Z

1

Just try specifying on only:

>>> df1.merge(df2, on='Site_ID').drop_duplicates()
   Site_ID       Town Business Name
0     1235    Fitzroy           BAC
1     2344  Glen Iris           RFG
>>>

edited Sep 1, 2021 at 6:15

answered Aug 27, 2021 at 2:02

U13-Forward

71.8k15 gold badges100 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

adey27 Over a year ago

Sorry about the late update on this. I did try above. but still I am getting duplicates

Collectives™ on Stack Overflow

How to avoid duplicate entries while joining two dataframes in Python?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related