Looping through dataframe list with IF Statement

Question

Just running a simple for-loop on a list of dataframes, however trying to add an IF clause... and it keeps erroring out.

df_list = [df1, df2, df3]
for df in df_list:
   if df in [df1, df2]:
      x = 1
   else:
      x = 2
.
.
.
ValueError: Can only compare identically-labeled DataFrame objects

Above is a simplified version of what I'm attempting. Can anyone tell me why this isn't working and a fix?

the error message is self-explanatory. Maybe try it with equality? — Srinivas Reddy Thatiparthy
– Srinivas Reddy Thatiparthy, Commented Mar 31, 2022 at 15:42
I smell an XY problem here. if statements in for loops are one thing, but your problem seems to come in when trying to evaluate if one dataframe is one of the dataframes in a list. Do you want to know if it is the same object? Or just has the same index, column, and values? Your question is ambiguous. — piRSquared
– piRSquared, Commented Mar 31, 2022 at 16:17

user7864386 · Accepted Answer · 2022-03-31 15:43:07Z

3

You could use DataFrame.equals with any instead:

df_list = [df1, df2, df3]
for df in df_list:
    if any(df.equals(y) for y in [df1, df2]):
        x = 1
    else:
        x = 2

answered Mar 31, 2022 at 15:43

user7864386

Sign up to request clarification or add additional context in comments.

Comments

Panwen Wang · Accepted Answer · 2022-03-31 16:48:50Z

2

Do NOT use .equals() here!

It's unnecessary and slowing down you program, use id() instead:

df_list = [df1, df2, df3]
for df in df_list:
   if id(df) in [id(df1), id(df2)]: 
      x = 1
   else:
      x = 2

Because here you just need to compare the identities, rather than the values.

edited Mar 31, 2022 at 16:48

answered Mar 31, 2022 at 16:38

Panwen Wang

3,8652 gold badges21 silver badges41 bronze badges

2 Comments

ALollz Over a year ago

Depending upon the use case id can be nice, but it could also be unwanted. For instance if df3 was created with df3=df1, then they share the same id, yet for some reason perhaps they should be handled differently. Guess that could be avoided with df3=df1.copy() so it's truly a different object, not just a reference

Panwen Wang Over a year ago

If df3=df1 is in the case, then @ALollz 's answer will be your choice. Neither id() nor .equals() can distinguish them. But id() is able to tell df3 from df1 if df3 is a copy of df1, while .equals() is not.

ALollz · Accepted Answer · 2022-03-31 15:52:59Z

2

You could use a better container and reference them by labels.

Equality checks for large DataFrames with object types can become slow, >> seconds, but it will take ~ns to check if the label is in a list.

dfs = {'df1': df1, 'df2': df2, 'df3': df3}
for label, df in dfs.items():
    if label in ['df1', 'df2']:
        x = 1
    else:
        x = 2

edited Mar 31, 2022 at 15:52

answered Mar 31, 2022 at 15:47

ALollz

59.7k7 gold badges73 silver badges97 bronze badges

Comments

mortice · Accepted Answer · 2022-03-31 15:46:52Z

0

You need to use df.equals()

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.equals.html

df_list = [df1, df2, df3]
for df in df_list:
   if df.equals(df1) or df.equals(df2):
      # blah blah

answered Mar 31, 2022 at 15:46

mortice

845 bronze badges

Comments

Muhammad Khuzaima Umair · Accepted Answer · 2022-03-31 15:54:17Z

0

The following link might help: Pandas "Can only compare identically-labeled DataFrame objects" error

According to this, the data frames being compared with == should have the same columns and index otherwise it gives the error.

Alternatively, you can compare the data frames using dataframe.equals method. Please refer to the documentation below: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.equals.html

answered Mar 31, 2022 at 15:54

Muhammad Khuzaima Umair

7368 silver badges20 bronze badges

Collectives™ on Stack Overflow

Looping through dataframe list with IF Statement

5 Answers 5

Comments

2 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

2 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related