Are there best practices for extensible magic methods in python?

Question

Say I have built a library containing a Foo class, with support for some magic methods, say __add__() and __radd__():

>>> class Foo(object):
...     def __add__(self, rhs):
...         print("Foo.__add__", rhs)
...     def __radd__(self, lhs):
...         print("Foo.__radd__", lhs)
... 
>>> foo = Foo()
>>> foo + 3
Foo.__add__ 3
>>> 3 + foo
Foo.__radd__ 3

When computing 3 + foo, python first calls type(3).__add__(3, foo), but as this returns NotImplemented, it falls back to type(foo).__radd__(foo, 3):

>>> type(3).__add__(3, foo)
NotImplemented

I would like developers to be able to build libraries on top of my library, say a library containing a class Bar, and I want them to have full control. In particular, I want to implement some mechanism that lets the other library decide whether foo + bar should call foo.__add__(bar) or bar.__radd__(foo).

I see that NumPy solved this using the __array_priority__ scheme. But this seems to cause some headaches (considering the number of questions and issues opened about this). Are there other best practices?

Although your question is quite interesting, it lacks details. Can you descrbe more aspects of desired behaviour? — Eli Korvigo
– Eli Korvigo, Commented Mar 9, 2017 at 21:55

MiniQuark · Accepted Answer · 2017-03-10 07:39:21Z

1

One popular option is to maintain a list of types supported by the LHS, and if the RHS's type is not in the list, then return NotImplemented:

class Foo(object):
    SUPPORTED_TYPES = (int, Foo)
    def __add__(self, rhs):
        if isinstance(type(rhs), SUPPORTED_TYPES):
            [...] # compute self + rhs
        else:
            return NotImplemented

This works well unless rhs is a smart subtype of one of the SUPPORTED_TYPES: there is no way it will get control. Moreover, listing types this way is not very flexible. It might be better to rely on duck typing than on a list of hard coded types.

answered Mar 10, 2017 at 7:39

MiniQuark

48.9k37 gold badges156 silver badges188 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

MiniQuark · Accepted Answer · 2017-03-10 08:35:42Z

A simple option is to try to let the LHS do whatever it needs to do (in the example below it calls the RHS's value() method) and in case it raises an exception, catch it and return NotImplemented:

class Foo(object):
    [...]
    def __add__(self, rhs):
        try:
            return self._value + rhs.value()
        except AttributeError:
            return NotImplemented

Simple as can be, no need to maintain a list of SUPPORTED_TYPES. However, there is a risk that the RHS implements a value() method that has nothing to do with this task, so it might be a bit risky. Moreover, there is no easy way for the rhs to get full control over the result.

In Python, it's usually better to beg for forgiveness rather than to ask for permission, as above, but you may prefer to check that the rhs has the value() method:

class Foo(object):
    def __add__(self, rhs):
        rhs_value_func = getattr(rhs, "value", None)
        if rhs_value_func is None:
            return NotImplemented
        else:
            return self._value + rhs_value_func()

MiniQuark · Accepted Answer · 2017-03-10 08:44:12Z

Yet another option is to use an attribute such as __foo_priority__, somewhat like NumPy does with its __array_priority__:

class Foo(object):
    __foo_priority__ = 0
    def __add__(self, rhs):
        delegate = True
        try:
            rhs_prio = type(rhs).__foo_priority__
            delegate = (self.__foo_priority__ < rhs_prio)
        except AttributeError:
            delegate = True
        if delegate:
            return NotImplemented
        else:
            return self.value_ + rhs.value()

This option is slightly more complicated, but pretty flexible. The only (minor) issue with this option is that it requires the rhs's type to have an extra attribute, so there's no way to give control to the rhs if it is an existing type without this attribute.

Collectives™ on Stack Overflow

Are there best practices for extensible magic methods in python?

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related