I tend to use smart pointers everywhere. I am not sure if this is a totally good idea, but I am lazy, and I cannot see any real downside [except if I wanted to do some C-style pointer arithmetic]. I use boost::shared_ptr because I know I can copy it around - if two entities share an image, then if one dies the other shouldn't lose the image too.

The downside of this is if one object deletes something it points to and owns, but something else is also pointing to it, then it isn't deleted.