How to filter a pandas dataframe using a variable filter

Question

I have a dataframe that contains multiple columns:

print(usersData):

    First name     Last name     ID1       ID2        Description         ...
0       Thomas        Shelby   TS123     T_SHE               Yeah         ...
1         Jack          Down   JD124     J_DOW               Yeah         ...
2         John           Doe   JD125     J_DOE               Yeah         ...
3        Alice          Lang   AL127     A_LAN               Yeah         ...

And I want to filter this filter based on variable criterias. I need to be able to filter using a combination of attributes that need to match a specific request. For example, if should be able to filter by First name AND Last name, like so:

firstname = "Thomas"
lastname = "Shelby"
searchResult = usersData[usersData['First name'] == firstname & usersData['Lastname'] == lastname]

But I want it to be dynamic, meaning I want to be able to do something like:

criterias = {"First name": "Thomas", "Last name": "Shelby"}
searchResult = usersData[for criteria in criterias: (usersData[criteria] == criterias[criteria] &)]

Anyone has an idea of the best way to do that?

Timeless · Accepted Answer · 2022-10-05 18:06:04Z

1

You can use boolean indexing with pandas.DataFrame.iloc and pandas.Series.isin combined.

criterias = {"First name": "Thomas", "Last name": "Shelby"}

usersData.loc[usersData[criterias.keys()].isin(criterias.values()).all(axis = 1),: ]

# Output :

 First name Last name    ID1    ID2 Description
0     Thomas    Shelby  TS123  T_SHE        Yeah

edited Oct 5, 2022 at 18:06

answered Oct 5, 2022 at 17:59

Timeless

38.3k6 gold badges33 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Bushmaster · Accepted Answer · 2022-10-05 18:14:09Z

0

eval() can be used:

import pandas as pd
df=pd.DataFrame(data={'first_name':['Thomas','Jack','John','Alice'],'last_name':['Shelby','Down','Doe','Lang'],
                      'ID1':['TS123','JD124','JD125','AL127']})

criterias = {"first_name": "Thomas", "last_name": "Shelby","ID1":"TS123"}

con=[]
for key, value in criterias.items():
    con.append("(df['{}']=='{}')".format(key,value))
    
condition=' & '.join(con)

print(condition)
'''
"(df['first_name']=='Thomas') & (df['last_name']=='Shelby') & (df['ID1']=='TS123')"
'''
final=df[eval(condition)]
print(final)

'''
      first_name last_name    ID1
0     Thomas     Shelby       TS123
'''

Can be developed according to the use case.

answered Oct 5, 2022 at 18:14

Bushmaster

4,6364 gold badges11 silver badges34 bronze badges

Comments

Eeshan · Accepted Answer · 2022-10-05 18:32:04Z

0

We can apply the filter conditions in a loop.

from copy import deepcopy
criteria = {"First name": "Thomas", "Last name": "Shelby"}
searchResult = deepcopy(usersData)
for c in criteria:
    searchResult = searchResult[searchResult[c] == criteria[c]]

answered Oct 5, 2022 at 18:32

Eeshan

995 bronze badges

Collectives™ on Stack Overflow

How to filter a pandas dataframe using a variable filter

3 Answers 3

# Output :

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

# Output :

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related