Pandas - Build A Query String, Use To Slice DataFrame

Question

I have a bunch of data stored in a DataFrame. I am trying to allows users to pass in query criteria in the form of:

column, operator, variable_name

So a user might pass in the following as an example

'Action equal allow,total_bytes > 10000,application neq facebook'

I parse that string by splitting and formatting into a query string that looks like this

query_string = (dframe['Action'] == 'allow') & (dframe['total_bytes'] > 10000) & ~(dframe[''Application] == 'facebook')

Then, I attempt to create a filtered table to return to the user by passing in the query_string I parsed the inputs to get.

dframe_filtered = dframe[query_string]

However this fails with a key error - I think because Python needs to see the query_string as not really a string - but Pandas series. Is there a way to make this work? Not sure you describe having Python not parse text as a string. But hopefully you all take the meaning.

Thanks!

Have you looked at df.query and take the user input in that format? — Jon Clements
– Jon Clements, Commented Oct 21, 2017 at 18:43

skrubber · Accepted Answer · 2017-10-21 23:49:00Z

1

A crude, quick and dirty way to convert the query string (assuming it will always be in this format) into machine-readable form is:

from functools import reduce
s = 'Action equal allow,total_bytes > 10000,Application neq facebook'
symbols = {'equal':'==', '>': '>', 'neq':'!=',',':'&'}

s1 = reduce(lambda x, y: x.replace(y, symbols[y]), symbols, s)
splits=s1.split('&')
splits1 = '('+splits[0].replace(splits[0].split()[2], '\''+splits[0].split()[2]+'\'')+')&('
splits2 = splits[1]
splits3 = ')&('+splits[2].replace(splits[2].split()[2],'\''+splits[2].split()[2]+'\')')
s2 = splits1+splits2+splits3
df1.query(s2)

    Action  Application total_bytes
0   allow   app1    11000
3   allow   app3    15000
4   allow   app5    17000

edited Oct 21, 2017 at 23:49

answered Oct 21, 2017 at 20:19

skrubber

1,1051 gold badge9 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pandas - Build A Query String, Use To Slice DataFrame

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related