10

I have some arguments taken from the user and passed along function to function (each function in a different class), until it eventually gets to a function that does some processing and then the solution is returned up the chain. Up the chain, the functions become more and more abstract merging results from multiple runs of the lower functions.

Where should I use *args and **kwargs?

I think *args and *kwargs can be used for every function where the function doesn't use the arguments explicitly. But, the actual arguments need to be defined at the top_level so that the user knows what the function expects.

Where should I define what the inputs mean?

I think they should be defined at the top_level because that's the one the end-user might want to see the documentation for.

Where should I define what default values?

Again, I think they should be defined at the top_level because that's the one the end-user interacts with.

This is a simple example to demonstrate the passing of the arguments, where I haven't shown how the functions become more and more abstract or how they interact with different classes, as I felt it was unnecessary detail.

def top_level(a=1, b=1, c=1, d=1, e=1):
    """ Compute sum of five numbers.
    :param a: int, a
    :param b: int, b
    :param c: int, c
    :param d: int, d
    :param e: int, e
    :return: int, sum
    """
    return mid_level(a, b, c, d, e)


def mid_level(*args, **kwargs):
    return bottom_level(*args, **kwargs)


def bottom_level(a, b, c, d, e):
    return a + b + c + d + e

print top_level(1, 2, 3)
8

Is there a Python convention for passing arguments like this?

4
  • I understand you're trying to make an example, but it's hard to think of an actual use-case is as mid_level is not currently serving any purpose. In fact, in that light, neither is bottom_level. Commented Mar 24, 2017 at 15:37
  • @Julien Yes, in a real environment, top_level might be the module that the user interfaces with, the mid_level the core module and 'bottom_level' a module that describes the data structures. Commented Mar 24, 2017 at 16:00
  • User interfaces with, as in from the command line via user input? Or from command arguments? Commented Mar 24, 2017 at 16:02
  • From a python console. The user might say import my_module, and then print top_level(1, 2, 3). Commented Mar 24, 2017 at 16:32

4 Answers 4

6

I'm not going to answer your question because it would be like answering the question "what's the best way to use a screwdriver to tighten a nut?". I.e. I do not believe that the tools you are asking for guidance with (*args and **kwargs) are designed to solve the problem you want to solve.

Instead I'll answer this question: "how do I associate some data with a set of functions?", and the answer to that is clearly Use Classes.

Welcome to object-oriented programming. I think you're going to enjoy it!

This is a very basic example of what I mean, but it was hard to know exactly what you wanted from your example since it was simple, but the basic principle is encapsulate your data in a class, and then operate on it using the class's methods.

  • You can then call between methods in the class without needing to pass loads of arguments around all the time (such as the .calculate() method below), which you don't know whether the top layer will need or a bottom layer.
  • You can just document the parameters in one place, the __init__ method.
  • You can customize through subclassing transparently to the code (because if you override a method in a subclass, it can still be used by the more generic superclass), as I've done for the .reduce(x, y) method below.

Example:

class ReductionCalculator:

    def __init__(self, *args):
        self.args = args

    def calculate(self):

        start = self.args[0]
        for arg in self.args[1:]:
            start = self.reduce(start, arg)

        return start


class Summer(ReductionCalculator):

    def reduce(self, x, y):
        return x + y


class Multiplier(ReductionCalculator):

    def reduce(self, x, y):
        return x * y

summer = Summer(1, 2, 4)
print('sum: %d' % (summer.calculate(),))
multiplier = Multiplier(1, 2, 4)
print('sum: %d' % (multiplier.calculate(),))
Sign up to request clarification or add additional context in comments.

2 Comments

I do already use classes passed along function to function (each function in a different class), and I do love them. In this situation, the functions that I'm passing arguments between are already in different classes.
OK, I see I didn't really take that bit in from your question... but if it's possible do you think you could provide a better example of what you're trying to achieve? Currently the example you've got doesn't make a lot of sense (and doesn't even use classes) and so it's hard to know what you intended. The key problem for me at the moment is that I can't picture your interfaces. Possibly I'm just trying to answer a question you don't feel you asked, in which case ignore me, but at the moment it just doesn't really smell right to me!
4

How about this approach: create a class, call it AllInputs, that represents the collection of all the "arguments taken from the user." The only purpose of this class is to serve as a container for a set of values. One instance of this class gets initialized, of course, at the top level of the program.

class AllInputs:
   def __init__(self,a=1, b=1, c=1, d=1, e=1):
    """ Compute sum of five numbers.
    :param a: int, a
    :param b: int, b
    :param c: int, c
    :param d: int, d
    :param e: int, e
    """
    self.a = a
    self.b = b
    self.c = c
    self.d = d
    self.e = e

This object, call it all_inputs, is now passed as the single argument to all of the functions in your example. If a function doesn't use any of the fields in the object, that's fine; it just passes it along to the lower-level function where the real work gets done. To refactor your example, you would now have:

def top_level(all_inputs):
    """ Compute sum of all inputs
    :return: int, sum
    """
    return mid_level(all_inputs)


def mid_level(all_inputs):
    return bottom_level(all_inputs)


def bottom_level(all_inputs):
    return (all_inputs.a + all_inputs.b + all_inputs.c + 
      all_inputs.d + all_inputs.e)

all_inputs = AllInputs(1, 2, 3)
print top_level(all_inputs)
8

I don't know if this is "Pythonic" or "non-Pythonic" and I don't care. I think it's a good programming idea to group together the data that the program will use. The initialization process, which combines default values with others taken from the user, is centralized in one place where it's easy to understand. It's reasonably self-documenting. You say the function calls are distributed across several classes, and that's no problem. The function calls are clean and the program flow is easy to follow. There is potential for optimization by placing some of the calculation inside AllInputs so you can avoid duplicating code.

What I don't like in your example (and I think you don't like it either, or you probably wouldn't have asked the question in the first place) is how it uses the *args syntax. When I see that syntax, I take it as a hint that all the arguments have the same semantic meaning, like in the standard library function os.path.join. In your application, if I understand the question, the low-level functions require the argument list to be in a specific order and have specific meanings (your example doesn't reflect that but the text suggests it). It's confusing to see arguments that get passed into a function as *args and then, at a lower level, their specific names and meanings appear once again. Grouping them into a single object makes it clear what's going on.

2 Comments

I think this hits the nail on the head. However, if the AllInputs class is used only as a container, I think a dictionary also serves the purpose. Maybe one final benefit to clearly point out is that when using the approach you describe, any change to a value in all_inputs (whether class or dict) is easily propagated up and down the levels.
I sometimes use small classes in cases where a dictionary would work just as well. That occasionally allows a static code checking tool (I use pylint) to catch an error when I type a variable name incorrectly. It can't do that with a dictionary.
2
+50

This isn't the most common pattern, but I've seen it for command line programs that have levels of nested commands: sub-commands, sub-sub-commands and so on. That's a model where "upper" level functions may be more or less dispatchers and not have information about what parameters are needed by the sub-functions within a given route. The purest scenario for this model is when the sub-commands are plugins and the "upper" layers have literally no information about the sub-functions, other than a calling convention the plug-ins are expected to adhere to.

In these cases, I'd argue the pythonic way is to pass parameters from higher-level to lower-level functions, and let the worker level decide which are useful. The range of possible parameters would be defined in the calling convention. This is pythonic on the basis of DRY -- don't repeat yourself. If the low-level / worker function defines what inputs are required or optional, it would often make sense to not repeat this information at the higher levels.

The same could be said for any inversion-of-control flow design, not just CLI applications w/ plug-ins. There are many application designs where I wouldn't use this approach, but it works here.

An input's meaning must be set at the topmost level it can arise in -- as an interface spec to lower levels (a convention, not programmatic). Otherwise the inputs would have no semantic meaning.

If an input can be used by multiple sub-functions, i.e. there's a chaining or pipeline concept in the control flow, then an input's default will also need to be defined at the topmost level for the input.

Comments

1

I would argue that passing arguments down several levels of functions is not pythonic in itself.

From the Zen of Python:

  • Simple is better than complex
  • Flat is better than nested

Edit:

If there are a lot of arguments and the functions inbetween just pass them down, I would probably wrap them up in a tuple and unwrap them at the lowest level.

4 Comments

But, at the same time functions should be short and do one thing only. I'm using multiple levels of functions because at each level, the function does something a little more abstract, as it operates on a more abstract class. And, it just so happens that I need to pass the same attributes between all of them.
The formulation of your question "until it eventually gets to a function that actually does some processing" implies that the functions inbetween do little if anything.
Ah I see, I will rephrase it.
...wrap them in a tuple... Let's call it "context" and throw all semantics over board. Sorry when that sounds harsh, but when facing that sort of issue I start refactoring immediately.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.