I'm trying to find some point of agreement with you.
I think from your comment, we can agree that there are two scenarios to consider: when the programmer is in control of the product and when the schedule is in control of the product. "Programming is not always so time-and-performance-sensitive, but it's not time-insensitive either."
Here's the deal: when you stop your design process and kick the compiler up and start coding, you haven't stopped your design work at all, you've simply switched from top-down to bottom-up design. Every line of code you write is a validation/denial of the assumptions and planning you're supposed to do before you start. If you're not refactoring aggressively as you go along, you're not taking the chance to see where your code is supposed to be going: you're simply trying as hard as you can to get to point B from point A. It's like driving a car from one side of the field to the other by letting go of the steering wheel and hoping it gets there okay. Most of the time you make it to the other side just fine, but you have zero control over where on the other side you're going to end up.
To me, that's hacking, and I mean in a bad way. I've been known to hack like a freaking electric weed-eater, trust me! But that's when something else is out of whack on the project. It's a sign that there are serious problems. When you're doing it right, your code is constantly evolving as you refactor to the point where changes become less and less of an issue. So good programming means you spend all the time necessary to refactor with the knowledge that you'll going to make it back in spades later on.
It's like when playing rummy, and you decide to pick up a huge stack of cards. At first, it seems like a crazy move, but if you know how to play the game, you realize that the only way to win is to constantly redefine the problem domain of what you have in your hand. Hacking doesn't get you that -- it just gets you to point B no matter what. It may not be the sign of poor programmer, but it's definitely a red light that something is out of whack somewhere.
My opinion only. Your mileage may vary. Do not attempt to drive heavy machinery after reading my comments. Consult a physician before beginning a regimen of refactoring. Etc.