initialization order in python class hierarchies

Question

In C++, given a class hierarchy, the most derived class's ctor calls its base class ctor which then initialized the base part of the object, before the derived part is instantiated. In Python I want to understand what's going on in a case where I have the requirement, that Derived subclasses a given class Base which takes a callable in its __init__ method which it then later invokes. The callable features some parameters which I pass in Derived class's __init__, which is where I also define the callable function. My idea then was to pass the Derived class itself to its Base class after having defined the __call__ operator

class Derived(Base):
    def __init__(self, a, b):
        def _process(c, d):
            do_something with a and b

        self.__class__.__call__ = _process
        super(Derived, self).__init__(self)

Is this a pythonic way of dealing with this problem?
What is the exact order of initialization here? Does one needs to call super as a first instruction in the __init__ method or is it ok to do it the way I did?
I am confused whether it is considered good practice to use super with or without arguments in python > 3.6

def __call__(self, ...): ... would definitely be preferable instead of self.__class__.__call__ — mcsoini
– mcsoini, Commented Dec 3, 2019 at 12:27
I cannot use the first syntax in your answer, since I need to construct the _process function within Derived classes init method — CD86
– CD86, Commented Dec 3, 2019 at 12:34
you are currently not using the parameters a and b at all. are these supposed to be passed to the Base class init? — mcsoini
– mcsoini, Commented Dec 3, 2019 at 12:35
no, they are just used for computations within the process function, Base is then passing c and d to process. I corrected the code above — CD86
– CD86, Commented Dec 3, 2019 at 12:37

bruno desthuilliers · Accepted Answer · 2019-12-03 14:51:21Z

3

What is the exact order of initialization here?

Well, very obviously the one you can see in your code - Base.__init__() is only called when you explicitely ask for it (with the super() call). If Base also has parents and everyone in the chain uses super() calls, the parents initializers will be invoked according to the mro.

Basically, Python is a "runtime language" - except for the bytecode compilation phase, everything happens at runtime - so there's very few "black magic" going on (and much of it is actually documented and fully exposed for those who want to look under the hood or do some metaprogramming).

Does one needs to call super as a first instruction in the init method or is it ok to do it the way I did?

You call the parent's method where you see fit for the concrete use case - you just have to beware of not using instance attributes (directly or - less obvious to spot - indirectly via a method call that depends on those attributes) before they are defined.

I am confused whether it is considered good practice to use super with or without arguments in python > 3.6

If you don't need backward compatibily, use super() without params - unless you want to explicitely skip some class in the MRO, but then chances are there's something debatable with your design (but well - sometimes we can't afford to rewrite a whole code base just to avoid one very special corner case, so that's ok too as long as you understand what you're doing and why).

Now with your core question:

class Derived(Base):
    def __init__(self, a, b):
        def _process(c, d):
            do_something with a and b

        self.__class__.__call__ = _process
        super(Derived, self).__init__(self)

self.__class__.__call__ is a class attribute and is shared by all instances of the class. This means that you either have to make sure you are only ever using one single instance of the class (which doesn't seem to be the goal here) or are ready to have totally random results, since each new instance will overwrite self.__class__.__call__ with it's own version.

If what you want is to have each instance's __call__ method to call it's own version of process(), then there's a much simpler solution - just make _process an instance attribute and call it from __call__ :

class Derived(Base):
    def __init__(self, a, b):
        def _process(c, d):
            do_something with a and b

        self._process = _process
        super(Derived, self).__init__(self)

   def __call__(self, c, d):
       return self._process(c, d)

Or even simpler:

class Derived(Base):
    def __init__(self, a, b):
        super(Derived, self).__init__(self)
        self._a = a
        self._b = b

   def __call__(self, c, d):
       do_something_with(self._a, self._b)

EDIT:

Base requires a callable in ins init method.

This would be better if your example snippet was closer to your real use case.

But when I call super().init() the call method of Derived should not have been instantiated yet or has it?

Now that's a good question... Actually, Python methods are not what you think they are. What you define in a class statement's body using the def statement are still plain functions, as you can see by yourself:

class Foo: ... def bar(self): pass ... Foo.bar

"Methods" are only instanciated when an attribute lookup resolves to a class attribute that happens to be a function:

Foo().bar main.Foo object at 0x7f3cef4de908>> Foo().bar main.Foo object at 0x7f3cef4de940>>

(if you wonder how this happens, it's documented here)

and they actually are just thin wrappers around a function, instance and class (or function and class for classmethods), which delegate the call to the underlying function, injecting the instance (or class) as first argument. In CS terms, a Python method is the partial application of a function to an instance (or class).

Now as I mentionned upper, Python is a runtime language, and both def and class are executable statements. So by the time you define your Derived class, the class statement creating the Base class object has already been executed (else Base wouldn't exist at all), with all the class statement block being executed first (to define the functions and other class attributes).

So "when you call super().__init()__", the __call__ function of Base HAS been instanciated (assuming it's defined in the class statement for Base of course, but that's by far the most common case).

edited Dec 3, 2019 at 14:51

answered Dec 3, 2019 at 13:54

bruno desthuilliers

78.3k6 gold badges103 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

CD86 Over a year ago

thank you for your elaborate answer, this is exactly what I am wondering about. Base requires a callable in ins init method. But when I call super().__init()__ the call method of Derived should not have been instantiated yet or has it?

mcsoini Over a year ago

@bruno desthuilliers out of curiosity: passing a callable child instance to the parent init strikes me as odd. would you see at that as a legitimate pattern? I don't know the wider context of the OT's problem, but in general calling a method in the parent class which is then overwritten by the children should do the job more cleanly... (?)

bruno desthuilliers Over a year ago

@mcsoini well, a callable is a callable, and Python doesn't care much about the concrete type. The nice point with passing callables instead of using inheritance is that it's much more flexible (cf the strategy pattern). So I see nothing wrong with this approach in and by itself - whether it's the best design for the task at hand is another question, but it depends on the context so there's no generic answer here.

CD86 Over a year ago

thank you for your amazing help! This becomes technically fast, as I can see xD. I have to get rid of the C++/statically-typed way of thinking in order to understand this, I guess

mcsoini Over a year ago

@bruno desthuilliers thanks for the reply, appreciate it!

|

Collectives™ on Stack Overflow

initialization order in python class hierarchies

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related