Higher order functions with varying parameters in Python

Question

I have the following code repeated over and over again:

def foo(data):
    result = (0, 0, 0)
    var1, var2, var3 = init_variables(data)
    for i in xrange(var1):
        result[0] += variable_func_1(data, var2, var3)
        result[1] += variable_func_2(data, var3)
        result[2] += fixed_func()
return result

Across all repetitions everything is fixed except for variable_func_n which can take a variable number of arguments, the number of these functions also varies.

I'd like to factor out the common code across all these functions so the first thing that comes to mind is to make use of higher order functions like so:

def foo(data, func_list):
    var1, var2, var3 = init_variables(data)
    results = (len(func_list) + 1) * [0]
    for i in xrange(var1):
        for j, func in enumerate(func_list):
            results[j] += (func(data, var1, var2, var3))
        results[len(func_list)] += fixed_func()
    return results

The problem with this solution is that it requires me to modify the signature of all the varying functions such that they can take more parameters. Is there a cleaner solution that wouldn't require me to modify the signature of all the passed in functions?

kwargs would require me to modify the signature of every function I pass in unfortunately. — Karl
– Karl, Commented Mar 1, 2016 at 22:15
Is there any regularity in the parameters taken by the varying functions? Like, any inspectable rule that says it should be variable_func_1(data, var2, var3) and not variable_func_1(data, var3, var2)? If there is no such rule, then I believe what you are asking for is not possible. Even inspect cannot tell which value should go to which position. — gil
– gil, Commented Mar 1, 2016 at 22:28
There is such a rule but I think relying on it will make for brittle code. inspect is probably going to be the best solution without refactoring I imagine. — Karl
– Karl, Commented Mar 1, 2016 at 22:37

qwwqwwq · Accepted Answer · 2016-03-01 22:10:50Z

1

Maybe you are looking for the inspect module?

In [11]: import inspect

In [12]: def f(a, b): pass

In [13]: inspect.getargspec(f)
Out[13]: ArgSpec(args=['a', 'b'], varargs=None, keywords=None, defaults=None)

This library can tell you the number of arguments in each function, as well as the name of each parameter, so you could then add your own logic to make sure the function is called correctly.

The real answer is that you might need to refactor your code, this pattern seems a little strange to me to be honest..

answered Mar 1, 2016 at 22:10

qwwqwwq

7,3592 gold badges32 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Jan Over a year ago

That is a cool function :o Didn't know that one. Take my upvote!

Karl Over a year ago

I agree that the pattern is a bit strange. For context the varying functions are particularly compute intensive and so I don't want to call them all when it wouldn't be necessary, they also return different values so I don't want to put them all within a big if/elif statement in one function and get that function to return differing values.

hruske Over a year ago

Using inspect is bad practice, a hack. If possible, you should strive to get your functions API such that it's useful without hacks, especially if you're thinking of using inspect on your code.

Jan · Accepted Answer · 2016-03-01 22:09:01Z

0

I think the easiest way to do this, is to simply pass a List that includes all data.

You could also give each function optional arguments but that would not help much

answered Mar 1, 2016 at 22:09

Jan

1,50410 silver badges15 bronze badges

Comments

hruske · Accepted Answer · 2016-03-01 22:37:03Z

0

Assuming you have a limited number of functions, how about creating an argument lookup table? It does insert an extra layer of redirection and duplicates some info, but then again it could replace a lot of other copypasted functions if I understand correctly:

func_sigs = {
    func1: ['data', 'var2', 'var3'],
    func2: ['data', 'var3']
}

def call_func(f, data, var1, var2, var3):
    args = [locals()[key] for key in func_sigs[f]]
    return f(*args)


def foo(data, func_list):
    var1, var2, var3 = init_variables(data)
    results = (len(func_list) + 1) * [0]
    for i in xrange(var1):
        for j, func in enumerate(func_list):
            results[j] += (call_func(func, data, var1, var2, var3))
        results[len(func_list)] += fixed_func()
    return results

The bigger question is why do you want to keep the signatures the way they are: are you keeping API compatibility or just don't want to refactor?

answered Mar 1, 2016 at 22:37

hruske

2,24319 silver badges27 bronze badges

1 Comment

Karl Over a year ago

I am willing to refactor, I just haven't thought of a compelling solution to this. All the varying functions require the information from init_variables but return different values and don't need to be called at the same time. (I'm working on a machine learning algorithm that optimizes subsets of parameters, the varying functions are actually functions that are used to calculate the gradients on those parameter subsets.)

Collectives™ on Stack Overflow

Higher order functions with varying parameters in Python

3 Answers 3

3 Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related