De-Facto cached property

[hughesadam_87]

Hi guys,

I've downloaded 2 cached_property implementations from the internet and both did not work correctly for my programs. Does anyone here used a cached_property function that works really well and could share with me?

[TrustyTony]

It would be usefull to know what you tried and the test which failed. I assume you are basically wanting memoize a property to an instance of the class. There would be many ways I think, some do not work in class without __dict__

https://github.com/mitsuhiko/werkzeug/blob/master/werkzeug/utils.py#L30

[hughesadam_87]

Thanks PyTony. I'll post the ones I tried, but first let me ask a clarifying question.

I'd like a property to implement the following behavior.

Say a property depends on two variables, a and b.

When I call the property, it returns a X b

If I call it again, it already has this value saved, so it merely returns it without recomputing a X b.

If a or b DO change and I call the property again, it will recompute the product a X b before returning it.

Some of the examples I'm seeing don't to the last part, aka recompute the value if the variable dependencies change.

I will try to implement the example you sent; however, it requires several module imports so I'll sift through all the source code there and take out the ones I need. But offhand, do you know if what I'm asking for is possible?

[Gribouillis]

Usually, I'm using this cached_property decorator.

Perhaps you could me more specific about your problem. A property usually means a computed attribute. I don't see an instance or an attribute in your post above, so you may be looking for something else.

[hughesadam_87]

Thanks Gribouillis. That property decorator looks nice and is what I'll use if I can modify it to incporporate the following behavior.

class X(object):
   def __init__(self):
       self.a=50
       self.b=30

   @cached_property
   def c:
      '''Return c from pythagorian theorem'''
      return=sqrt(self.a**2 + self.b**2)

I'd like it to cache its value just like you code does; however, if I were to change a or b (eg:

self.a=52

I'd like the cache to be emptied and then recomputed the next time c is called. In essence, the property can check if any dependent variables have changed. If so, it updates c, otherwise it uses cached values. Basically, it's a "smart" cache.

Does behavior like this exist somehow? I know it can be implemented using properties in the enthought tool suite for example... but these are special object types whose purpose is to implement such behavior.

[Gribouillis]

One way is to store a pair (key, value) instead of a value. When the attribute is accessed, the key is computed. If the key is different from the previous one, the value is recomputed. Here is an example

from collections import namedtuple
from math import sqrt

class cached_property_with_key(object):

    Record = namedtuple("Record", "key value")

    def __init__(self, func, name = None, key_func = None):
        self.func = func
        self.__name__ = "_cache_" + (name or func.__name__)
        self.key_func = key_func

    def __get__(self, instance, klass=None):
        if instance is None:
            return self
        key = self.key_func(instance) if self.key_func else None
        record = instance.__dict__.get(self.__name__, None)
        if not(record and record.key == key):
            record = self.Record(key, self.func(instance))
            instance.__dict__[self.__name__] = record
        return record.value

    def set_key(self, func):
        self.key_func = func

class X(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b

    @cached_property_with_key
    def c(self):
        print("X.c computed")
        return sqrt(self.a**2 + self.b**2)

    @c.set_key
    def _c_key(self):
        return (self.a, self.b)

if __name__ == "__main__":
    x = X(0,0)
    print(x.c)
    print(x.c)
    x.a = 3
    print(x.c)
    x.b = 3
    print(x.c)
    print(x.c)
    print(x.__dict__)

""" my output -->
X.c computed
0.0
0.0
X.c computed
3.0
X.c computed
4.24264068712
4.24264068712
{'a': 3, 'b': 3, u'_cache_c': Record(key=(3, 3), value=4.242640687119285)}
"""

Notice that I can't store the cached pair under the same name as the property in the instance's dict because the __get__ method must be called every time x.c is invoked.

[hughesadam_87]

Awesome man, thanks for posting this. Let me play with it and bring back some feedback soon.

[Gribouillis]

In this second version, I'm using a generator to implement the cached property with key: the method X.c() yields the key first, then the value. If the key is the same as the previous one, the code after the first yield is not executed. I think it's a better implementation.

from collections import namedtuple
from math import sqrt

class cached_property_with_key2(object):
    Record = namedtuple("Record", "key value")

    def __init__(self, func, name = None):
        self.func = func
        self. __name__ = name if name else ("_cache_" + func.__name__)

    def __get__(self, instance, klass=None):
        if instance is None:
            return self
        gen = self.func(instance)
        key = next(gen)
        record = instance.__dict__.get(self.__name__, None)
        if not(record and record.key == key):
            record = instance.__dict__[self.__name__] = self.Record(key, next(gen))
        return record.value

class X(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b

    @cached_property_with_key2
    def c(self):
        yield (self.a, self.b)
        print("X.c computed")
        yield sqrt(self.a**2 + self.b**2)

[hughesadam_87]

Sorry for the late reply. This is exactly what I needed and works great. I honestly think this is the best implementation of property for costly operations. Perhaps thinks of a catchier named than cached_property_with_key2 and put it as a code snippet? I'd like to save this and use it in many future applications, so is there a way you'd like to be credited besided your daniweb account name?