Use the right tool for the job.

If your program can throw exceptions ensure your code is exception aware. Using smart pointers, RAII, and avoiding 2 phase construction are good starting points.

If you have cyclic references with no clear ownership semantics you can consider using a garbage collection library or refactoring your design.

Good libraries will allow you to code to the concept not the type so it shouldn't matter in most cases which kind of pointer you are using beyond resource management issues.

If you're working in a multi-threaded environment, make sure you understand if your object is potentially shared across threads. One of the main reasons to consider using boost::shared_ptr or std::tr1::shared_ptr is because it uses a thread-safe reference count.

If you're worried about the separate allocation of the reference counts there are many ways around this. Using the boost::shared_ptr library you can pool allocate the reference counters or use boost::make_shared (my preference) which allocates the object and the reference count in a single allocation thereby alleviating most cache miss concerns people have. You can avoid the performance hit of updating the reference count in performance critical code by holding a reference to the object at the topmost level and passing around direct references to the object.

If you need shared ownership but don't want to pay the cost of reference counting or garbage collection consider using immutable objects or a copy on write idiom.

Bear in mind that far and away your biggest performance wins are going to be at an architecture level, followed by an algorithm level, and while these low level concerns are very important they should be tackled only after you've addressed the major issues. If you're dealing with performance issues at the level of cache misses then you have a whole host of issues that you also have to be aware of like false sharing which have nothing to do with pointers per say.

If you're using smart pointers just to share resources like textures or models consider a more specialized library like Boost.Flyweight.

Once the new standard becomes adopted move semantics, rvalue references, and perfect forwarding will make working with expensive objects and containers much easier and more efficient. Until then don't store pointers with destructive copy semantics, such as auto_ptr or unique_ptr, in a Container (the standard concept). Consider using the Boost.Pointer Container library or storing shared ownership smart pointers in Containers. In performance critical code you can consider avoiding both of these in favor of intrusive containers such as those in Boost.Intrusive.

The target platform shouldn't really influence your decision too much. Embedded devices, smart phones, dumb phones, PCs, and consoles can all run the code just fine. Project requirements such as strict memory budgets or no dynamic allocation ever/after load are more valid concerns and should influence your choices.