pythonic way to apply multiple filters to a dataframe based on user input

Question

Suppose I have a dataframe of celebrities with their age, ethnicity, height, industry etc.

I want to create a function where I can filter the dataframe systematically such that multiple filters can be applied.

e.g

def filter_data(df, filter_col, filter_val, filter_amount):
    if filter_amount == 1:
        df = df[df.filter_col[0] == filter_val[0]]
    if filter_amount == 2:
        df = df[(df.filter_col[0] == filter_val[0]) & (df.filter_col[1] == filter_val[1])]

    etc

Where filter_col is a list of columns for which you wish to filter by, and filter_val is also a list of values and filter_amount is an integer

I want it to be systematical so that for any filter amount it would proceed to filter the dataset based on the values of the list without having to manually code it out

help.

look into pandas.Dataframe.query

jkr
– jkr

2020-07-11 19:58:51 +00:00
Commented Jul 11, 2020 at 19:58 — jkr
– jkr, Commented Jul 11, 2020 at 19:58

Zishaan Sunderji · Accepted Answer · 2020-07-11 20:07:44Z

1

Since the filter performs an and (&), it would make sense to do it like this:

import pandas as pd

def filter_data(df, filter_col, filter_val, filter_amount):
    out = df.copy()
    for i in range(filter_amount):
        out = out[out[filter_col[i]] == filter_val[i]]
    return out

def main():
    x = pd.DataFrame({"Age": [12, 44, 23], "Ethnicity": ["White", "Black", "White"], "Height": [180, 182, 168]})
    #    Age Ethnicity  Height
    # 0   12     White     180
    # 1   44     Black     182
    # 2   23     White     168

    y = filter_data(x, ["Ethnicity", "Height"], ["White", 180], 1)
    #    Age Ethnicity  Height
    # 0   12     White     180
    # 2   23     White     168

    z = filter_data(x, ["Ethnicity", "Height"], ["White", 180], 2)
    #    Age Ethnicity  Height
    # 0   12     White     180

answered Jul 11, 2020 at 20:07

Zishaan Sunderji

1861 gold badge1 silver badge6 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

score 0 · Accepted Answer · 2020-12-07 03:31:49Z

0

filter_vals = [1, 2, 3]
filter_amount = 3

filtered_df = [df[df[col] == val] for val in filter_vals]

edited Dec 7, 2020 at 3:31

answered Dec 6, 2020 at 23:58

user14776666

2 Comments

am.rez Over a year ago

shouldn't it be like this: filtered_df = [df[df[col] == val] for val in filter_vals]? current code will produce a syntax error

user14776666 Over a year ago

yes, I forgot the brackets, added them now

Collectives™ on Stack Overflow

pythonic way to apply multiple filters to a dataframe based on user input

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related