Understanding class variable behavior

Question

We came across the need to have a dynamic class variable in the following code in python 2.

from datetime import datetime
from retrying import retry

class TestClass(object):
    SOME_VARIABLE = None

    def __init__(self, some_arg=None):
        self.some_arg = some_arg

    @retry(retry_on_exception=lambda e: isinstance(e, EnvironmentError), wait_fixed=3000 if SOME_VARIABLE == "NEEDED" else  1000, stop_max_attempt_number=3)
    def some_func(self):
        print("Running {} at {}".format(self.some_arg, datetime.now()))
        if self.some_arg != "something needed":
            raise EnvironmentError("Unexpected value")


TestClass.SOME_VARIABLE = "NEEDED"
x = TestClass()
x.some_func()

Output:

Running None at 2021-07-26 19:40:22.374736
Running None at 2021-07-26 19:40:23.376027
Running None at 2021-07-26 19:40:24.377523
Traceback (most recent call last):
  File "/home/raj/tmp/test_test.py", line 19, in <module>
    x.some_func()
  File "/home/raj/.local/share/virtualenvs/test-DzpjW1fZ/lib/python2.7/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/home/raj/.local/share/virtualenvs/test-DzpjW1fZ/lib/python2.7/site-packages/retrying.py", line 212, in call
    raise attempt.get()
  File "/home/raj/.local/share/virtualenvs/test-DzpjW1fZ/lib/python2.7/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/home/raj/.local/share/virtualenvs/test-DzpjW1fZ/lib/python2.7/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/home/raj/tmp/test_test.py", line 14, in some_func
    raise EnvironmentError("Unexpected value")
EnvironmentError: Unexpected value

We can see that the value of SOME_VARIABLE is not being updated.

Trying to understand if there is way in which we can update SOME_VARIABLE dynamically. The use case is to have dynamic timings in the retry function based on SOME_VARIABLE value at runtime.

The arguments to the decorator are computed immediately while the class is being defined, not when you call the method. — chepner
– chepner, Commented Jul 26, 2021 at 14:22
The decorator gets called while the class body is being evaluated, before the class object is finally created and definitely before you execute TestClass.SOME_VARIABLE = .... — chepner
– chepner, Commented Jul 26, 2021 at 14:25
yeah, makes sense now. Looks like I overlooked the concept of decorator evaluation. Thanks for the info. — Raj
– Raj, Commented Jul 26, 2021 at 14:30
I've expanded the explanation and provided a possible workaround in an answer. — chepner
– chepner, Commented Jul 26, 2021 at 14:33

chepner · Accepted Answer · 2021-07-26 14:32:59Z

Your class definition is equivalent, based on the definition of decorator syntax, to

class TestClass(object):
    SOME_VARIABLE = None

    def __init__(self, some_arg=None):
        self.some_arg = some_arg

    decorator = retry(retry_on_exception=lambda e: isinstance(e, EnvironmentError),
                      wait_fixed=3000 if SOME_VARIABLE == "NEEDED" else  1000,
                      stop_max_attempt_number=3)

    def some_func(self):
        ...

    some_func = decorator(some_func)

Note that retry is called long before you change the value of TestClass.SOME_VARIABLE (indeed, before the class object that will be bound to TestClass even exists), so the comparison SOME_VARIABLE == "NEEDED" is evaluated when SOME_VARIABLE still equals None.

To have the retry behavior configured at run-time, try something like

class TestClass(object):
    SOME_VARIABLE = None

    def __init__(self, some_arg=None):
        self.some_arg = some_arg

    def _some_func_implemenation(self):
        print("Running {} at {}".format(self.some_arg, datetime.now()))
        if self.some_arg != "something needed":
            raise EnvironmentError("Unexpected value")

    def some_func(self):
        wait = 3000 if self.SOME_VARIABLE == "NEEDED" else 1000
        impl = retry(retry_on_exception=lambda e: isinstance(e, EnvironmentError), 
                     wait_fixed=wait,
                     stop_max_attempt_number=3)(self._some_func)
        return impl()

some_func becomes a function that, at runtime, creates a function (based on the private _some_func) with the appropriate retry behavior, then calls it.

(Not tested; I may have gotten the interaction between the bound method self._some_func and retry wrong.)

Working perfectly. A minor typo - it is self._some_func_implemenation instead of self._some_func in some_func.

Collectives™ on Stack Overflow

Understanding class variable behavior

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related