10

Is there any method I can override that will allow me to use print statements / pdb / etc. to keep track of every time an instance of my class is allocated? While unpickling some objects I am seeming to get some that never have either __setstate__ or __init__ called on them. I tried overriding __new__ and printing out the id of every object I make in __new__, but I am still encountering objects with ids that were never printed.

Edit: here is my code I use for altering (instrumenting) __new__ of my class and all of its super-classes except for object itself:

class Allocator:
    def __init__(self, my_class):
       self.my_class = my_class
       self.old_new = my_class.__new__

    def new(self, * args, ** kargs):
        rval = self.old_new(*args, ** kargs)
        #rval = super(self.my_class,cls).__new__(cls)
        print 'Made '+str(self.my_class)+' with id '+str(id(rval))
        return rval

def replace_allocator(cls):
    if cls == object:
        return

    setattr(cls,'__new__',Allocator(cls).new)
    print cls.__base__

    try:
        for parent in cls.__base__:
            replace_allocator(parent)
   except:
        replace_allocator(cls.__base__)

I call replace_allocator on my classes' parent class as soon as it is imported in the main script. My class has a custom __new__ to begin with, which also prints out the id.

Improve this question
7
  • Are you absolutely certain they are different instances? Have you checked if id(instance) matches? Commented Jan 25, 2011 at 20:18
  • Are you trying to use Allocator(cls) as a type cast? If so, it is wrong, because in Python it is a constructor call for an Allocator object, and the .new part will get a new bound-method instance. Commented Jan 25, 2011 at 23:15
  • Rosh: Yes. My allocator function shows the way that I print out id(instance). Later, when I find an object that is missing some fields that should have been added by setstate or init, I print out its id too. If I grep for that id in the earlier printouts there are no hits. Commented Jan 25, 2011 at 23:49
  • Apalala: It is indeed my intention to use Allocator(cls) as a constructor call for an Allocator object. The point of an Allocator constructed with cls is to call the original __new__ method of cls and report the id of the resulting object instance. Commented Jan 25, 2011 at 23:51
  • The iteration should be for parent in cls.__bases__. You don't need the fall-back to cls.__base__. And don't use catch-all except: clauses. They always bite you. Commented Jan 26, 2011 at 12:04

3 Answers 3

Reset to default
5

(This is more of a comment than an answer.)

Quoting Guido's Unifying types and classes in Python 2.2:

There are situations where a new instance is created without calling __init__ (for example when the instance is loaded from a pickle). There is no way to create a new instance without calling __new__ (although in some cases you can get away with calling a base class's __new__).

If you are using new-style classes (descendants of object), __new__() should always be called. I don't think the obscure cases "you can get away with calling a base class's __new__" in will happen accidently, though I don't know what these cases actually are.

And just to add an example:

In [1]: class A(object):
   ...:     def __new__(cls):    
   ...:         print "A"
   ...:         return object.__new__(cls)
   ...:     

In [2]: A()
A
Out[2]: <__main__.A object at 0xa3a95cc>

In [4]: object.__new__(A)
Out[4]: <__main__.A object at 0xa3a974c>
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

This sounds like a feasible approach but unfortunately if you can get away with calling the base class, then that might explain why there are objects getting allocated without my knowledge. I'm only allowed to override __new__ for classes other than object itself.
0

Are you using new-style classes? For Pickle to call __setstate__, the __getstate__ method should also be defined on the class returning a non-False value.

Improve this answer

1 Comment

This does not answer the question of how to detect objects as they are created. But yes I am using new-style classes and I do have a __getstate__ implemented. Also, __setstate__ is getting called on most but not all instances of the class.
0

Not sure if this can help you, but Python's garbage collector has introspective capabilities that might be worth taking a look at.

Improve this answer

2 Comments

The question is about keeping track of the creation of objects.
You're right, I most probably misunderstood the question. But now I wonder, won't the garbage collector need itself to monitor object allocation? If so, one might devise a method to hook into the gc and figure out when an object of interested gets allocated. I'm not questioning what you said, I'm just curious.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.