How to merge multiple DataFrames in python

Question

I have a list of Dataframes and Im trying to merge it into a one using the _id column.

List of dataframes(df) looks like

0  thyx07y1bg8   ...     2000-03-31   2004-12-31
1   ofr7s6wf1j   ...     2000-03-31   2004-12-31
    
[2 rows x 4 columns],            _id    ... calculate_from calculate_to
0  3sw1btgso6t   ...     2000-03-31   2004-12-31
1   ofr7s6wf1j   ...     2000-03-31   2004-12-31
[2 rows x 4 columns],            _id    ... calculate_from calculate_to

the result im expecting is

0  thyx07y1bg8    ...     2000-03-31   2004-12-31
1   ofr7s6wf1j    ...     2000-03-31   2004-12-31
2  3sw1btgso6t    ...     2000-03-31   2004-12-31
[3 rows x 4 columns],            _id   ... calculate_from calculate_to

I have tried

pd.concat(df)

and

reduce(lambda left, right: pd.merge(left, right, on=["_id"], how="inner"),df)

but couldnt get the result I want. any idea ? Thanks

BENY · Accepted Answer · 2022-11-15 22:37:53Z

2

This is more like combine_first or concat with drop

out = pd.concat([df1, df2]).drop_duplicates('id')

Or

out = df1.set_index('id').combine_first(df2.set_index('id')).reset_index()

answered Nov 15, 2022 at 22:37

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to merge multiple DataFrames in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related