how to filter dataframe using a function?

Question

i would like to use the following function to filter a dataframa

def isInRadius(position):
    latCheck = False
    lonCheck = False
    if position.lat < 0:
        latCheck = position.lat <= upperLat and position.lat >= lowerLat
    else:
        latCheck = position.lat >= upperLat and position.lat <= lowerLat

    if not latCheck:
        return False

    if position.lon < 0:
        lonCheck = position.lon <= righterLon and position.lon >= lefterLon
    else:
        lonCheck = position.lon >= righterLon and position.lat <= lefterLon

    return latCheck and lonCheck

The dataframe has more columns than 'lat' and 'lon' but i would like to filter it by those 2 according to the logic implemented on the function above.

I have try dataFrame.filter(lambda x: isInRadius(x)) and dataFrame.filter(isInRadius) and dataFrame.filter(lambda x: isInRadius(x.iLoc[0])) and other approaches but none worked, resulting in the error "TypeError: 'function' object is not iterable"

How should i do it?

On C# i would do

var filtered = myCollection.Where(x => isInRadius(x));

what's lowerLat, upperLat, etc? Can you explain the logic of isInRadius? — Quang Hoang
– Quang Hoang, Commented Aug 1, 2019 at 18:54
filter works on index, so it should not work regardless of what syntax you use since the filtering you want is done over the values and not the indexes — Yuca
– Yuca, Commented Aug 1, 2019 at 18:56
does it really matter? upperLat, lowerLat, righterLon and lefterLon defines a box on the surface of the earth and i wanna know if the position provided is inside it... — Leonardo
– Leonardo, Commented Aug 1, 2019 at 18:56
pd.DataFrame.filter is used to filter the dataframe based on the index or column header labels. You need to create a boolean series using your function then use .loc. — Scott Boston
– Scott Boston, Commented Aug 1, 2019 at 18:57
Try something like df[df.apply(lambda x: isInRadius(x), 1)] — pythonic833
– pythonic833, Commented Aug 1, 2019 at 19:05

pythonic833 · Accepted Answer · 2019-08-01 19:38:16Z

4

Just use the .apply functionality of pandas dataframes

df[df.apply(isInRadius, 1)]

edited Aug 1, 2019 at 19:38

answered Aug 1, 2019 at 19:22

pythonic833

3,2341 gold badge16 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

MichaelD · Accepted Answer · 2019-08-01 20:00:15Z

0

No need to use an apply

import pandas as pd

df2 =pd.DataFrame({'lookup_id':['a','a','c','c','c'],'val':[1,1,1,1,1]})
print(df2)

  lookup_id  val
0         a    1
1         a    1
2         c    1
3         c    1
4         c    1

# Replace this with whatever your function actually does
def val_is_a(r):
    return r['lookup_id'] =='a'

df2.loc[lambda df : val_is_a(df)]

    lookup_id   val
0   a           1
1   a           1

answered Aug 1, 2019 at 20:00

MichaelD

1,3361 gold badge11 silver badges17 bronze badges

Collectives™ on Stack Overflow

how to filter dataframe using a function?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related