Python: changing methods and attributes at runtime

This example shows the differences between adding a method to a class and to an instance.

>>> class Dog():
...     def __init__(self, name):
...             self.name = name
...
>>> skip = Dog('Skip')
>>> spot = Dog('Spot')
>>> def talk(self):
...     print 'Hi, my name is ' + self.name
...
>>> Dog.talk = talk # add method to class
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk()
Hi, my name is Spot
>>> del Dog.talk # remove method from class
>>> skip.talk() # won't work anymore
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'
>>> import types
>>> f = types.MethodType(talk, skip, Dog)
>>> skip.talk = f # add method to specific instance
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk() # won't work, since we only modified skip
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'

A possibly interesting alternative to using types.MethodType in:

>>> f = types.MethodType(talk, puppy, Dog)
>>> puppy.talk = f # add method to specific instance

would be to exploit the fact that functions are descriptors:

>>> puppy.talk = talk.__get__(puppy, Dog)

I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?

You can add and remove attributes and methods to any class, and they'll be available to all instances of the class:

>>> def method1(self):
       pass

>>> def method1(self):
       print "method1"

>>> def method2(self):
       print "method2"

>>> class C():
       pass

>>> c = C()
>>> c.method()

Traceback (most recent call last):
  File "<pyshell#62>", line 1, in <module>
    c.method()
AttributeError: C instance has no attribute 'method'

>>> C.method = method1
>>> c.method()
    method1
>>> C.method = method2
>>> c.method()
    method2
>>> del C.method
>>> c.method()

Traceback (most recent call last):
  File "<pyshell#68>", line 1, in <module>
    c.method()
AttributeError: C instance has no attribute 'method'
>>> C.attribute = "foo"
>>> c.attribute
    'foo'
>>> c.attribute = "bar"
>>> c.attribute
    'bar'

you can just assign directly to the class (either by accessing the original class name or via __class__ ):

class a : pass
ob=a()
ob.__class__.blah=lambda self,k: (3, self,k)
ob.blah(5)
ob2=a()
ob2.blah(7)

will print

(3, <__main__.a instance at 0x7f18e3c345f0>, 5)
(3, <__main__.a instance at 0x7f18e3c344d0>, 7)

Simply:

f1 = lambda:0                   #method for instances
f2 = lambda _:0                 #method for class
class C: pass                   #class

c1,c2 = C(),C()                 #instances

print dir(c1),dir(c2)

#add to the Instances
c1.func = f1
c1.any = 1.23

print dir(c1),dir(c2)
print c1.func(),c1.any

del c1.func,c1.any

#add to the Class
C.func = f2
C.any = 1.23

print dir(c1),dir(c2)
print c1.func(),c1.any
print c2.func(),c2.any

which results in:

['__doc__', '__module__'] ['__doc__', '__module__']
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__']
0 1.23
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__', 'any', 'func']
0 1.23
0 1.23

another alternative, if you need to replace the class wholesale is to modify the class attribute:

>>> class A(object):
...     def foo(self):
...         print 'A'
... 
>>> class B(object):
...     def foo(self):
...         print 'Bar'
... 
>>> a = A()
>>> a.foo()
A
>>> a.__class__ = B
>>> a.foo()
Bar

Does the class itself necessarily need to be modified? Or is the goal simply to replace what object.method() does at a particular point during runtime?

I ask because I sidestep the problem of actually modifying the class to monkey patch specific method calls in my framework with getattribute and a Runtime Decorator on my Base inheritance object.

Methods retrieved by a Base object in getattribute are wrapped in a Runtime_Decorator that parses the method calls keyword arguments for decorators/monkey patches to apply.

This enables you to utilize the syntax object.method(monkey_patch="mypatch"), object.method(decorator="mydecorator"), and even object.method(decorators=my_decorator_list).

This works for any individual method call (I leave out magic methods), does so without actually modifying any class/instance attributes, can utilize arbitrary, even foreign methods to patch, and will work transparently on sublcasses that inherit from Base (provided they don't override getattribute of course).

import trace

def monkey_patched(self, *args, **kwargs):
    print self, "Tried to call a method, but it was monkey patched instead"
    return "and now for something completely different"

class Base(object):

    def __init__(self):
        super(Base, self).__init__()

    def testmethod(self):
        print "%s test method" % self

    def __getattribute__(self, attribute):
        value = super(Base, self).__getattribute__(attribute)
        if "__" not in attribute and callable(value):
            value = Runtime_Decorator(value)
        return value

class Runtime_Decorator(object):

    def __init__(self, function):
        self.function = function

    def __call__(self, *args, **kwargs):

        if kwargs.has_key("monkey_patch"):
            module_name, patch_name = self._resolve_string(kwargs.pop("monkey_patch"))
            module = self._get_module(module_name)
            monkey_patch = getattr(module, patch_name)
            return monkey_patch(self.function.im_self, *args, **kwargs)

        if kwargs.has_key('decorator'):
            decorator_type = str(kwargs['decorator'])

            module_name, decorator_name = self._resolve_string(decorator_type)
            decorator = self._get_decorator(decorator_name, module_name)
            wrapped_function = decorator(self.function)
            del kwargs['decorator']
            return wrapped_function(*args, **kwargs)

        elif kwargs.has_key('decorators'):
            decorators = []

            for item in kwargs['decorators']:
                module_name, decorator_name = self._resolve_string(item)
                decorator = self._get_decorator(decorator_name, module_name)
                decorators.append(decorator)

            wrapped_function = self.function
            for item in reversed(decorators):
                wrapped_function = item(wrapped_function)
            del kwargs['decorators']
            return wrapped_function(*args, **kwargs)

        else:
            return self.function(*args, **kwargs)

    def _resolve_string(self, string):
        try: # attempt to split the string into a module and attribute
            module_name, decorator_name = string.split(".")
        except ValueError: # there was no ".", it's just a single attribute
            module_name = "__main__"
            decorator_name = string
        finally:
            return module_name, decorator_name

    def _get_module(self, module_name):
        try: # attempt to load the module if it exists already
            module = modules[module_name]
        except KeyError: # import it if it doesn't
            module = __import__(module_name)
        finally:
            return module

    def _get_decorator(self, decorator_name, module_name):
        module = self._get_module(module_name)
        try: # attempt to procure the decorator class
            decorator_wrap = getattr(module, decorator_name)
        except AttributeError: # decorator not found in module
            print("failed to locate decorators %s for function %s." %\
            (kwargs["decorator"], self.function))
        else:
            return decorator_wrap # instantiate the class with self.function

class Tracer(object):

    def __init__(self, function):
        self.function = function

    def __call__(self, *args, **kwargs):
        tracer = trace.Trace(trace=1)
        tracer.runfunc(self.function, *args, **kwargs)

b = Base()
b.testmethod(monkey_patch="monkey_patched")
b.testmethod(decorator="Tracer")
#b.testmethod(monkey_patch="external_module.my_patch")

The downside to this approach is getattribute hooks all access to attributes, so the checking of and potential wrapping of methods occurs even for attributes that are not methods + won't be utilizing the feature for the particular call in question. And using getattribute at all is inherently somewhat complicated.

The actual impact of this overhead in my experience/for my purposes has been negligible, and my machine runs a dual core Celeron. The previous implementation I used introspected methods upon object init and bound the Runtime_Decorator to methods then. Doing things that way eliminated the need to utilize getattribute and reduced the overhead mentioned previously... however, it also breaks pickle (maybe not dill) and is less dynamic then this approach.

The only use cases I have actually come across "in the wild" with this technique were with timing and tracing decorators. However, the possibilities it opens up are extremely wide ranging.

If you have a preexisting class that cannot be made to inherit from a different base (or utilize the technique it's own class definition or in it's base class'), then the whole thing simply does not apply to your issue at all unfortunately.

I don't think setting/removing non-callable attributes on a class at runtime is necessarily so challenging? unless you want classes that inherit from the modified class to automatically reflect the changes in themselves as well... That'd be a whole 'nother can o' worms by the sound of it though.