As the quote says, many programmers make the mistake of (trying to) build a system, not a game. Typically that system keeps ballooning out of control until it's so complex that theoretically it can handle anything, but in practicality all you have is a big bundle of code. Or more often, before you even get to a working stage, you are so tangled up in code that doesn't run that you lose focus (if you had any to begin with) and motivation (since nothing is truly working).
Prototypes and iteration tend to work much better. In the end, a great design might come out of it, but more often something more simple and refined comes out of it. KISS and YAGNI come to mind.
I personally believe there needs to be a balance. If there's a core mechanic of your game, work on it. You still need to iterate, but you do need to refine it. Hint: organization of your code is not a core mechanic of your game.
Case in point: Peggle, by PopCap Games. A core mechanic of the game is the ball physics. They perfected it! I'm sure they spent quite a lot of time making sure it was absolutely perfect, because it is what makes the game. But at the same time, I can totally picture an early prototype of their game that maybe just draws sprites to the screen and does some type of more primitive collision detection and bouncing, just to see if the game idea is fun. Then once they found out that shooting a ball can actually be fun, they refined the bouncing of the ball. (this is all just speculation, of course)
It also depends on your technical requirements, which you should nail down early on. The platform that your game runs on should not change, or if it does, you need to know exactly how you plan to allow it to change, no more and no less. Design on that. If you're developing a game for OpenGL, and you don't care about DirectX, then really, don't care about it. That means that if it's more convenient for you to have each entity draw itself and not worry about Factories and other design patterns like that, then do that. It's okay, because it meets the requirements. You should not have to change it later, despite what you tell yourself. And really, worst case scenario? Refactor later. It takes time later but it lets you focus on the now, getting a working game on one platform even if it means it can't simultaneously and automatically run on your toaster.
Test driven design, however, is a more opinionated topic. I am of the belief that game developers should do more of it. I also think game developers have some of the most rigorous, tight schedules, and they think they can't afford to spend time on TDD when they just want to get a game going. Also, again with the motivation, TDD is a lot slower and you get to see a lot less of a functioning game (in the beginning at least). This can have serious effects on programmer motivation.
I think it's also just a general lack of knowledge. I don't think TDD is prevalent in other areas either, but like agile development, I think it's spreading. You might say it's ahead of its time (or maybe not, as the case may be years from now). More important than TDD is "RDD" - Requirements Driven Development. I just made that up. What is your goal? To make a game. Everything else comes second. If you can prove that TDD increases productivity and helps teams reach deadlines, then don't you think everyone would be using it? And maybe that's the case. But right now, our industry is more competitive than ever, there are harder and sooner deadlines, and stuff just needs to work. Construction workers don't build scaffolding first; they lay a foundation, then they raise some walls and floors, and only then do they build selective bits of scaffolding to do specific tasks which scaffolding makes more convenient. I think the same applies to software development.
Sorry for such a long post. I hope you've gained some bits of wisdom from it. I am just a student, talking about my observations, with very limited industry experience but a lot of reading from industry experts. So take my words with a grain of salt.
And hey, do what you think will work. You can always change it or scrap it and start over. That's the cool thing about any form of engineering; if at first you don't succeed, try something different. (or something like that :-P ) You're not pouring concrete; software is malleable.