Python: Cache values of descriptors

Question

I would like to clarify some things about Python's descriptors. I want to add a property to my class with some complex set/get mechanics and cache those calculated values inside the descriptor object. A simplified example looks like this:

class Pro(object):
    """My descriptor class"""

    def __init__(self):
        self.value = None

    def __get__(self, instance, owner):
        return self.value

    def __set__(self, instance, value):
        self.value = value


class Something(object):
    """My simple class"""
    pro = Pro()

a = Something()
a.pro = 1

b = Something()
b.pro = 2

print(a.pro, b.pro) # At first, I've expected them to be 1 and 2

I thought somehow that pro attribute would be unique instance of Pro for every instance of Something, obviously I was wrong. Looks like I should use something like instance._value instead of self.value inside __set__ and __get__, but I've really hoped to hide everything inside Pro class. Is this even possible? Thanks!

@Lemming, looks pretty neat, thanks, but I was hoping to solve this with standard library only :) — Anton Melnikov
– Anton Melnikov, Commented Jan 29, 2015 at 13:21
Fair enough. Btw, as most python packages this one is open source. The actual code is fairly small and the non-threading version doesn't need any extra packages. — Lemming
– Lemming, Commented Jan 29, 2015 at 18:01
The source was interesting, thanks, haven't seen it until now. — Anton Melnikov
– Anton Melnikov, Commented Jan 29, 2015 at 18:36

Ashwini Chaudhary · Accepted Answer · 2015-01-29 13:39:52Z

4

The problem with your code is that you're setting the attribute on Pro's instance which is going to shared by all instances of Something. To fix this you should set an attribute on the individual instance of Something, one way to do that is to use a metaclass:

class Meta(type):
    def __new__(cls, name, bases, dct):
        for k, v in dct.items():
            if isinstance(v, Pro):
                # add an _ in front of the name
                v.name = '_' + k
        return super(Meta, cls).__new__(cls, name, bases, dct)


class Pro(object):

    def __get__(self, ins, typ):
        return getattr(ins, self.name)

    def __set__(self, ins, val):
            setattr(ins, self.name, val)

class Something(object):
    """My simple class"""
    __metaclass__ = Meta
    pro = Pro()

a = Something()
a.pro = 1

b = Something()
b.pro = 2

Demo:

>>> a.pro, b.pro
(1, 2)
>>> a.__dict__
{'_pro': 1}
>>> b.__dict__
{'_pro': 2}
>>> a.pro = 100
>>> a.__dict__
{'_pro': 100}

So there is no way around making hidden attributes in Something instances, right?

No, there is. You can store a dictionary in Pro's instance that stores all the values related to each instance of Something. For example if Something's instances are hashable then you can do something like this using weakref.WeakKeyDictionary. The WeakKeyDictionary will make sure that that once a Something's instance has no references left then it is garbage collected immediately which is not possible with a normal dict:

from weakref import WeakKeyDictionary

class Pro(object):

    def __init__(self):
        self.instances = WeakKeyDictionary()

    def __get__(self, ins, typ):
        return self.instances[ins]

    def __set__(self, ins, val):
        self.instances[ins] = val

p = Pro()

class Something(object):
    """My simple class"""
    pro = p

a = Something()
a.pro = 1

b = Something()
b.pro = 2

print a.pro, b.pro

print p.instances.items()
del a
print p.instances.items()

Output:

1 2
[(<__main__.Something object at 0x7fb80d0d5310>, 1), (<__main__.Something object at 0x7fb80d0d5350>, 2)]
[(<__main__.Something object at 0x7fb80d0d5350>, 2)]

edited Jan 29, 2015 at 13:39

answered Jan 29, 2015 at 13:12

Ashwini Chaudhary

252k60 gold badges478 silver badges519 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Anton Melnikov Over a year ago

So there is no way around making hidden attributes in Something instances, right?

Geoff Genz Over a year ago

If you want to 'cache' the value for an instance, somewhere that value has got to be associated with the instance. Python attributes basically do that by using the dict dictionary of the object. You could reverse it by having a dictionary in the description with the instance as the key, but the basic problem remains that you need a mapping of instance to value or vice versa

Ashwini Chaudhary Over a year ago

@AntonMelnikov Definitely that is possible as well, see the update.

Collectives™ on Stack Overflow

Python: Cache values of descriptors

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related