Comparing columns of different pandas dataframes

Question

I'm new to Python and using pandas dataframes to store and work with a large dataset.

I'm interested in knowing whether it's possible to compare values between dataframes of similarly named columns. For example, the functionality I'm after would be similar to comparing the column 'A' in this dataframe:

to the column 'A' in this one:

Then, for each row I would determine which of the two 'A' values is smaller and add it to, say, a new column in the first dataframe called 'B':

I'm aware of the

pandas.DataFrame.min

method but as I understand it this will only located the smallest value of one column and can't be used to compare columns of different dataframes. I'm not sure of any other ways in which this functionality could be achieved

Any suggestions for solving this (probably) very simple question would be much appreciated! Thank you.

akuiper · Accepted Answer · 2016-11-14 16:05:46Z

1

You can use numpy.minimum():

import numpy as np
df1['B'] = np.minimum(df1.A, df2.A)

Or use Series.where() to replace values:

df1['B'] = df1['A'].where(df1.A < df2.A, df2.A)

answered Nov 14, 2016 at 16:05

akuiper

216k33 gold badges362 silver badges379 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Sam L. Over a year ago

Great, thank you for that. I used the Series.where() method. It works for most of my dataset, but in some cases I have dataframes that are of unequal length. Do you think that is likely to be the cause of this error message? ValueError: Can only compare identically-labeled Series objects Is there a workaround for dealing with dataframes of unequal lengths that you know of?

akuiper Over a year ago

How would you like the result to be if the data frames are of unequal lengths?

Sam L. Over a year ago

Good point. Having thought about it: I'm looking to still select the smallest value even if it is located in a row that does not occur in the other DataFrame. Perhaps now though, it is best to write the smallest value to a new Series or DataFrame since it could be that the DataFrame we were previously adding to is the shorter one. I hope this is all clearly described, please let me know if I can clarify anything.

Collectives™ on Stack Overflow

Comparing columns of different pandas dataframes

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related