Python attrs class attribute cached lazy load

Question

I have classes that look like this:

@attr.s
class ImageMagic(object):
    path = attr.ib()

    _img = attr.ib()

    @_img.default
    def _img(self):
        return Image.open(self.path)

@attr.s
class FileObject(object):
    # Standard
    path = attr.ib()

    # When magic-ed
    magic = attr.ib(default=None)

My goal is to have attrs.asdict() to be able to serialize the FileObject by going through all the attrs and initializing the magic attribute only when it's actually called for serialization and not on __init__.

As most of the time I really don't want the Magic library to be inspecting the object as it is an expensive IO operation.

Goal:
a) How to connect the two classes

b) have the magic attribute only instantiate the ImageMagic object when I actually call it.

c) Only once, so that it may be reused later on if called multiple times.

With this I would prefer to use the Attrs library.

General unclean solution would be to have a @property with a getter, getter checks for existence of private _magic attribute and loads if it doesn't exist.

And then somehow registers the property to attrs library so that it may get serialized furthermore.

This is an example of how an actual solution might look like:

@attr.s
class IOExpensiveClass(object):
    path = attr.ib()

    _hash = attr.ib()

    @_hash.default
    def _img(self):
        return IOOPERATION(self.path)


@attr.s
class FileObject(object):
    # Standard
    path = attr.ib()

    _magic = None

    # Missing attrs registration, that I yet don't know how to write
    @property
    def magic(self):
        return self._magic or IOExpensiveClass(self.path)

another not so clean solution would be to override getattr, capture the access to magic and do the lazy evaluation there. — Chobeat
– Chobeat, Commented Apr 11, 2019 at 7:19
You could use something like this for lazy attribute evaluation: gist.github.com/waszil/171d64e5b94cdd51c404d41332f476c0 — waszil
– waszil, Commented Dec 6, 2021 at 6:45

hynek · Accepted Answer · 2019-04-23 10:21:31Z

Your question when taking a step back is one of serialization. Python's way for lazy-loading is indeed using properties so you got that part right.

The problem of serialization (and more so deserialization) is one that keeps coming up a lot both here and on the attrs bug tracker. The problem is that it's an inherently complex topic which is why at some point we decided to keep it out of scope except for simple cases (i.e. straight asdict/astuple) and let the community come up with specialized libraries.

And indeed there is a bunch of libraries for [sd]erialization that you can find in the attrs wiki but I don't know whether or not any of them supports your valid-but-edgy use case.

That said, if neither fixes your need, your use case can totally be achieved using attrs' extension machinery and metadata. I'm not sure if you can bend asdict to your will, but in the worst case you can just copy/paste it and add your own logic. The function is quite simple.

Collectives™ on Stack Overflow

Python attrs class attribute cached lazy load

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related