Reformulation of a lambda function with functools.partial in Python

Question

I currently have the following structure:

Inside a class I need to handle several types of functions with two special variables and an arbitrary number of parameters. To wrap these for the methods I apply them on I scan the function signatures first (that works very reliable) and decide what the parameters and what my variables are.

I then bind them back with a lambda expression in the following way. Let func(x, *args) be my function, then I'll bind

f = lambda x, t: func(x=x, **func_parameter)

In the case that I get func(x, t, *args) I bind

f = lambda x, t: func(x=x, t=t, **func_parameter)

and similar if I have neither variables. It is essential that I hand a function of the form f(x,t) to my methods inside that class.

I would like to use functools.partial for that - it is the more pythonic way to do it and the performance when executing is better (the function f is potentially called a couple of million times...). The problem that I have is that I don't know what to do if I have a basis function which is independent of one of the variables t and x, that's why I went with lambda functions at all, they just map the other variable 'blind'. It's still two function calls and while definitions with lambda and partial take the same time, execution is a lot faster with partial.

Does anyone knoe how to use partial in that case? Performance is kind of an issue here.

EDIT: A little later. I figured out that function evaluation with tuple arguments are faster than with keyword arguments, so that was a plus. And then, in the end, as a user I would just take some of the guess work from Python, i.e. directly define def func(x): return 2*x instead of def func(x, a): return a*x And call it directly. In that way I can use the function directly. Second case would be if I implement the case where x and t are both present as partial mapping. That might be a compromise.

I'm not sure I get what you mean by "a basis function which is independent of one of the variables". Can't you just add x and/or t to func_parameter and then do functools.partial(**func_parameter)? — BrenBarn
– BrenBarn, Commented Jul 23, 2017 at 18:07
Sure. Let's assume the 'base function' is func(x, a, b). If I map it to parameters as described above, I still get f(x, t) as desired. How can I get that constellation with functools.partial in this case? As you see t is not part of the function signature. With lambda I can formulate that conveniently. — freistil90
– freistil90, Commented Jul 23, 2017 at 18:12
Do you intend to make a bunch of partial functions in advance so you don't have to create them at run time? — wwii
– wwii, Commented Jul 23, 2017 at 18:23
Can you elaborate that a little? Not sure what you mean @wwii — freistil90
– freistil90, Commented Jul 23, 2017 at 18:39
It might not be relevant but do you create the functions on the fly while the process is running or are they created in advance as part of the static code? — wwii
– wwii, Commented Jul 23, 2017 at 18:42

wwii · Accepted Answer · 2017-07-24 03:07:42Z

1

You could write adapter classes that have an f(x,t) call signature. The result is similar to functools.partial but much more flexible. __call__ gives you a consistent call signature and lets you add, drop, and map parameters. Arguments can be fixed when an instance is made. It seems like it should execute as fast as a normal function, but I have no basis for that.

A toy version:

class Adapt:
    '''return a function with call signature f(x,t)'''
    def __init__(self, func, **kwargs):
        self.func = func
        self.kwargs = kwargs
    def __call__(self, x, t):
        # mapping magic goes here
        return self.func(x, t, **self.kwargs) 
        #return self.func(a=x, b=t, **self.kwargs)

def f(a, b, c):
    print(a, b, c)

Usage:

>>> f_xt = Adapt(f, c = 4)
>>> f_xt(3, 4)
3 4 4
>>>

Don't know how you could make that generic for arbitrary parameters and mappings, maybe someone will chime in with an idea or an edit.

So if you end up writing an adapter specific to each function, the function can be embedded in the class instead of an instance parameter.

class AdaptF:
    '''return a function with call signature f(x,t)'''
    def __init__(self, **kwargs):
        self.kwargs = kwargs
    def __call__(self, x, t):
        '''does stuff with x and t'''
        # mapping magic goes here
        return self.func(a=x, b=t, **self.kwargs)
    def func(self, a, b, c):
        print(a, b, c)

>>> f_xt = AdaptF(c = 4)
>>> f_xt(x = 3, t = 4)
3 4 4
>>>

I just kinda made this up from stuff I have read so I don't know if it is viable. I feel like I should give credit to the source I read but for the life of me I can't find it - I probably saw it on a pyvideo.

.

edited Jul 24, 2017 at 3:07

answered Jul 24, 2017 at 2:51

wwii

23.9k7 gold badges42 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

wwii Over a year ago

Rats ... I imagine a function factory wouldn't be any different than your lambdas either.

freistil90 Over a year ago

What I did notice though is that executing functions via positional arguments ( with *args ) is substantially faster than with keywords ( with **kwargs ). So for performance, keep that in mind....

Collectives™ on Stack Overflow

Reformulation of a lambda function with functools.partial in Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related