Can you dynamically combine multiple conditional functions into one in Python?

Question

I'm curious if it's possible to take several conditional functions and create one function that checks them all (e.g. the way a generator takes a procedure for iterating through a series and creates an iterator).

The basic usage case would be when you have a large number of conditional parameters (e.g. "max_a", "min_a", "max_b", "min_b", etc.), many of which could be blank. They would all be passed to this "function creating" function, which would then return one function that checked them all. Below is an example of a naive way of doing what I'm asking:

def combining_function(max_a, min_a, max_b, min_b, ...):
    f_array = []
    if max_a is not None:
        f_array.append( lambda x: x.a < max_a )
    if min_a is not None:
        f_array.append( lambda x: x.a > min_a )
    ...

    return lambda x: all( [ f(x) for f in f_array ] )

What I'm wondering is what is the most efficient to achieve what's being done above? It seems like executing a function call for every function in f_array would create a decent amount of overhead, but perhaps I'm engaging in premature/unnecessary optimization. Regardless, I'd be interested to see if anyone else has come across usage cases like this and how they proceeded.

Also, if this isn't possible in Python, is it possible in other (perhaps more functional) languages?

EDIT: It looks like the consensus solution is to compose a string containing the full collection of conditions and then use exec or eval to generate a single function. @doublep suggests this is pretty hackish. Any thoughts on how bad this is? Is it plausible to check the arguments closely enough when composing the function that a solution like this could be considered safe? After all, whatever rigorous checking is required only needs to be performed once whereas the benefit from a faster combined conditional can be accrued over a large number of calls. Are people using stuff like this in deployment scenarios or is this mainly a technique to play around with?

Are all your conditional parameters going to be max/min pairs? It's definitely possible, but how to do it depends on what kind of tests you'll be doing. — David Z
– David Z, Commented Jun 9, 2010 at 22:18
Remove the list in the call to all(), i.e. just all (f(x) for f in f_array). This will stop early if any f produces a false value. — user319799
– user319799, Commented Jun 9, 2010 at 22:20
@doublep: this will work only with python recent enough. Some people still have to work with archaic versions ;-) — liori
– liori, Commented Jun 9, 2010 at 22:21
@liori: OK, you have a point. Then ditching the lambda and returning a local function definition that does for f in f_array: ... will be more efficient. The current version never stops early and creates a list object that's not required. — user319799
– user319799, Commented Jun 9, 2010 at 22:25
The spec is unclear. seems like first you want to check min_a < x.a < max_a but for the next pair check some other attribute, presumably x.b? And since args don't come with their names, no way to know what's the next argument to check! — Nas Banov
– Nas Banov, Commented Jun 10, 2010 at 0:11

user319799 · Accepted Answer · 2010-06-09 22:41:45Z

Replacing

return lambda x: all( [ f(x) for f in f_array ] )

with

return lambda x: all( f(x) for f in f_array )

will give a more efficient lambda as it will stop early if any f returns a false value and doesn't need to create unnecessary list. This is only possible on Python 2.4 or 2.5 and up, though. If you need to support ancient values, do the following:

def check (x):
    for f in f_array:
        if not f (x):
            return False
    return True

return check

Finally, if you really need to make this very efficient and are not afraid of bounding-on-hackish solutions, you could try compilation at runtime:

def combining_function (max_a, min_a):
    constants = { }
    checks    = []

    if max_a is not None:
        constants['max_a'] = max_a
        checks.append ('x.a < max_a')

    if min_a is not None:
        constants['min_a'] = min_a
        checks.append ('x.a > min_a')

    if not checks:
        return lambda x: True
    else:
        func = 'def check (x): return (%s)' % ') and ('.join (checks)
        exec func in constants, constants
        return constants['check']

class X:
    def __init__(self, a):
        self.a = a

check = combining_function (3, 1)
print check (X (0)), check (X (2)), check (X (4))

Note that in Python 3.x exec becomes a function, so the above code is not portable.

David Z · Accepted Answer · 2010-06-09 22:37:50Z

1

Based on your example, if your list of possible parameters is just a sequence of max,min,max,min,max,min,... then here's an easy way to do it:

def combining_function(*args):
    maxs, mins = zip(*zip(*[iter(args)]*2))
    minv = max(m for m in mins if m is not None)
    maxv = min(m for m in maxs if m is not None)
    return lambda x: minv < x.a < maxv

But this kind of "cheats" a bit: it precomputes the smallest maximum value and the largest minimum value. If your tests can be something more complicated than just max/min testing, the code will need to be modified.

edited Jun 9, 2010 at 22:37

answered Jun 9, 2010 at 22:28

David Z

133k29 gold badges264 silver badges284 bronze badges

3 Comments

David Z Over a year ago

Oops, I missed the .a in the question... I'll edit accordingly.

Nas Banov Over a year ago

well here lies a problem, the spec is unclear. seems like you need to check min_a < x.a < max_a but for the next pair check some other attribute, presumably 'x.b'. And since args don't come with their names, no way to know what's the next argument to check!

David Z Over a year ago

@EnTerr: Of course if I'd interpreted the question that way I would have written something else. You're right that it's unclear.

jfs · Accepted Answer · 2010-06-09 23:15:47Z

1

The combining_function() interface is horrible, but if you can't change it then you could use:

def combining_function(min_a, max_a, min_b, max_b):
    conditions = []
    for name, value in locals().items():
        if value is None:
            continue
        kind, sep, attr = name.partition("_")
        op = {"min": ">", "max": "<"}.get(kind, None)
        if op is None:
            continue
        conditions.append("x.%(attr)s %(op)s %(value)r" % dict(
            attr=attr, op=op, value=value))

    if conditions:
        return eval("lambda x: " + " and ".join(conditions), {})
    else:
        return lambda x: True

edited Jun 9, 2010 at 23:15

answered Jun 9, 2010 at 23:06

jfs

417k210 gold badges1k silver badges1.7k bronze badges

6 Comments

liori Over a year ago

combining_function(None, None, None, type('', (), {'__repr__': lambda x:'os.system("sudo rm -rf /")'})()) ;-)

erich Over a year ago

The interface was just demonstration purposes, what would you suggest as an ideal interface for this kind of problem?

jfs Over a year ago

@liori: I don't see your point. You could just as easily write type('',(),{'__lt__': lambda x,y: os.system("whatever")})().

jfs Over a year ago

@erich: If there are many min/max pairs then an incremental improvement to the interface might be get_inrange(a=(None,1), b=(1, 2)) ->

get_inrange = lambda **kwargs: lambda x: all(op(getattr(x, attr), value) for attr, minmax in kwargs.items() for op, value in zip([operator.gt, operator.lt], minmax) if value is not None)

liori Over a year ago

@J.F. Sebastian: my point is that this code needs to check its arguments or clearly state that it is unsafe for some inputs; otherwise "eval" could do anything.

|

Collectives™ on Stack Overflow

Can you dynamically combine multiple conditional functions into one in Python?

3 Answers 3

Comments

3 Comments

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related