How to dynamically loop values to filter pandas dataframe?

Question

I need to loop my df.columns so that i can dynamically apply condition to my every column through loop, i am not able to loop my column names in condition.

df:

   B1    B2    B3   B4   B5
   9     0     5    6    7
   8     7     6    4    8
   0     9     8    6    6
   1     0     7    6    3

condition = [(df['B1'] == 0)| (df['B1'].isnull==False),
             (df['B1'] == 8)| (df['B1'].isnull==True),
             (df['B1'] == 6)| (df['B1'].isnull==False)]

values = [999,444,555]

I used to do this:

df['B1'] = np.select(condition , values) # Seperately for every column.

I am trying:

for i in df.columns:
    df[i] = np.select(condition, values) # how can i able to loop i in condition, since condition is constant

Output for B1:

   B1    B2    B3   B4   B5
   999   0     5    6    7 
   999   7     6    4    8
   999   9     8    6    6
   999   0     7    6    3

Could you please describe you desired output? How ahould the result look like? — mosc9575
– mosc9575, Commented Sep 22, 2021 at 14:41
Every single value has to go through condition, which ever condition satisfies, it will change value. # of condition should be equal to # of values always. — James Lin
– James Lin, Commented Sep 22, 2021 at 15:30

Cimbali · Accepted Answer · 2021-09-22 14:48:36Z

1

You can pass a dictionary to .eq() to check equality of different columns with different values:

>>> df.eq({'B1': 0, 'B2': 7, 'B3': 6, 'B4': 4, 'B5': 7})
      B1     B2     B3     B4     B5
0  False  False  False  False   True
1  False   True   True   True  False
2   True  False  False  False  False
3  False  False  False  False  False

Similarly you can then do that after .isna():

>>> df.isna().eq({'B1': False, 'B2': False, 'B3': True, 'B4': False, 'B5': False})
     B1    B2     B3    B4    B5
0  True  True  False  True  True
1  True  True  False  True  True
2  True  True  False  True  True
3  True  True  False  True  True

And then finally combine both with |

answered Sep 22, 2021 at 14:48

Cimbali

11.5k1 gold badge44 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mosc9575 · Accepted Answer · 2021-09-23 06:49:52Z

0

One possible way is to use mask(), this replaces a value, where a condition is True. If you loop over your conditions and values, I think you get, what you need.

conditions = [(df == 0)| (df.isnull==False),
             (df == 8)| (df.isnull==True),
             (df == 6)| (df.isnull==False)]

values = [999,444,555]        
    
for condition, value in zip(conditions, values):
    df.mask(condition, value, inplace=True)

The output for your example is this

    B1   B2   B3   B4   B5
0    9  999    5  555    7
1  444    7  555    4  444
2  999    9  444  555  555
3    1  999    7  555    3

answered Sep 23, 2021 at 6:49

mosc9575

6,3922 gold badges12 silver badges36 bronze badges

Collectives™ on Stack Overflow

How to dynamically loop values to filter pandas dataframe?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related