Differing Behavior With Pandas Boolean Operation

Question

For the following test code, could someone explain why the contents of test1 and test2 differ for the third row (index 2)? I had code written a la test1 and wasted a few hours trying to figure out why the code wasn't working before re-writing a la test2 to solve it. Is it an order-of-operations thing within pandas for the one-liner used to generate test1?

import pandas as pd

# test data
dat1 = [1,1,2,1]
dat2 = ['a','b','c','d'] # only for building a multi-column DF, not otherwise used
dat3 = [False,False,True,False]

# build a dataframe
df = pd.DataFrame( data={ "Data1":dat1, "Data2":dat2})

# first test
test1 = df['Data1']==1 | pd.Series(dat3)
print('==test 1==')
print(test1)

# second test
tmp1 = df['Data1']==1
tmp2 = pd.Series(dat3)
test2 = tmp1 | tmp2
print('==test 2==')
print(test2)

The output is

==test 1==
0     True
1     True
2    False
3     True
dtype: bool
==test 2==
0    True
1    True
2    True
3    True
dtype: bool

anky · Accepted Answer · 2021-01-14 18:24:42Z

1

The difference is caused due to operator precedence,try:

test1 = (df['Data1']==1) | pd.Series(dat3) # Note the parenthesis (df['Data1']==1)

Then you would see both are equal.

# first test
test1 = (df['Data1']==1) | pd.Series(dat3)
print('==test 1==')
print(test1)

# second test
tmp1 = df['Data1']==1
tmp2 = pd.Series(dat3)
test2 = tmp1 | tmp2
print('==test 2==')
print(test2)

==test 1==
0    True
1    True
2    True
3    True
dtype: bool
==test 2==
0    True
1    True
2    True
3    True
dtype: bool

When you do:

df['Data1']==1 | pd.Series(dat3)

This is evaluated as:

df['Data1'] == (1 | pd.Series(dat3))

To avoid this use parenthesis for each condition:

test1 = (df['Data1']==1) | pd.Series(dat3)

edited Jan 14, 2021 at 18:24

answered Jan 14, 2021 at 18:17

anky

75.3k11 gold badges46 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

HeorotsHero Over a year ago

Thank you for the clarifications and explanation on how things are evaluated, I figured it must be something simple that like.

Collectives™ on Stack Overflow

Differing Behavior With Pandas Boolean Operation

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related