Python: make lib use redefined class

Question

imagine using a python library. Suppose that this library has one script and a couple of classes that are of interest. All these classes are defined in individual files. Suppose that class D is imported and used by class A,B and C in multiple ways. In the same way class A uses class B and class B uses class C. The script makes direct use of class A and thus direct and indirect use of class D. Now suppose that you want to customize D and locally define a class D' that is derived from class D.

What would be the most elegant way to make the script and the other classes (A,B, C) use class D' instead of D?

Thank you and have a nice day!

There is not general answer, what are you attempting to do specifically? — Olivier Melançon
– Olivier Melançon, Commented Mar 4, 2019 at 18:25
This is totally going to depend on how exactly the imports are managed. For example, this will be annoyingly difficult if the scripts use from d import D instead of import d and then in the files use d.D to reference D. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Mar 4, 2019 at 18:26
For common libraries such as json you can inherit the classes and then use that class in function calls, e.g. json.dump(x, fp, encoder=YourNewClass) — Rocky Li
– Rocky Li, Commented Mar 4, 2019 at 18:30

munk · Accepted Answer · 2019-03-04 18:55:05Z

1

You can monkey patch the library.

# a.py
import d
class A():
    my_d = d.D()

# script.py
A.d.D = D_prime

def foo():
    a_cls = a.A() # uses A.d.D is D'

The way that a.py imports D matters

# a.py
from d import D
class A():
    my_d = D()

# script.py
A.D = D_prime

def foo():
    a_cls = a.A() # uses A.D is D'

Other import schemes may involve similar patterns. Something like this might be very difficult to patch.

def builder():
    from d import D
    return D()

It may also be helpful to see how the mock library does this. If it's a toy, you could use this directly. patch in particular is interesting.

@patch('A.d.D', new=D_prime)
def my_func(...):
    a = A() # a.d.D is D_prime

Monkey patching is a code smell, depending on unittest in application code is a smell. None of it is "elegant". If you are the library author, please support dependency injection instead.

If you're brave, you might pull out some of the patch decorator into something that's not focused on mocks. Because this pattern exists in the unittest library, it can be considered pythonic and "elegant" with the caveat above.

If you're interested in how this works, you're modifying the symbol table for the module a. See globals and locals for details.

edited Mar 4, 2019 at 18:55

answered Mar 4, 2019 at 18:32

munk

13.2k10 gold badges54 silver badges72 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

DisplayName Over a year ago

Thank you (and everyone else). Once I got my head around it, it turned out to be quite easy. For anyone else having problems to understand it, here is a less abstract example that I found once I knew the word monkey patch: (stackoverflow.com/questions/19545982/…) .

DisplayName Over a year ago

Follow up: why does my monkey patch break down? The lib I'm using is ray 0.6.2 (github.com/ray-project/ray) and I succesfully applied my patch like this: ray.tune.trial.Trial.init_logger = custom_init_logger. So far, so good. Now I want to also patch another function ray.rllib.evaluation.metrics.summarize_episodes = custom_summarize_episodes, which doesnt work. Working back I found that while ray.rllib.agents.dqn.DQNAgent.train = exit works, ray.tune.trainable.Trainable.train = exit doesnt. 'Trainable' is imported and it's train function called in the agents train function.

munk Over a year ago

@DisplayName I suspect what's happening is ray.rllib.agents.dqn.DQNAgent.train is imported from ray.tune.trainable.Trainable.train. If the patch happens after this import, and you've patched ray.tune.trainable.Trainable.train, the place where it's used has a reference to the original function.

Collectives™ on Stack Overflow

Python: make lib use redefined class

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related