I am a workstation user.
I've used both Mac Pro and HP Z800 workstations, and currently use a HP Z800 that is probably 3 or 4 years old, but also have used a Mac Pro as recently as 2 years ago.
I've never upgraded a CPU in a workstation.
The CPUs I buy are chosen for performance at a sane (relative) price. I'm currently running a couple of Xeon X5560 CPUs, I think they cost £1,600 each at the time.
I also buy as much RAM as is reasonable for the box, I started this Z800 on 24GB ECC RAM which was dirt cheap (relatively - as many slots on the motherboard meant I could get lots of the smaller sizes which were cheap). When the price later dropped on the larger modules I upped it to 192GB RAM.
As for graphics card... now that is something I upgrade every 2 years or so. I started on a Quadro FX1800 and am now on a Quadro 6000.
Aren't the names of these things wonderful? You know you get more when the number is HUGE!
I think this is basically the norm: Upgrade RAM, upgrade GPU... leave CPU until you replace the whole box.
PS: And whilst I'm here, the Mac Pro is a good design... but for the wrong product. I think this is a new design that is the right design with airflow, cooling and silence in mind and would suit desktop users well... but desktop requirements are not workstation requirements and not being able to replace GPUs is a non-starter. I'll skip this design of Mac Pro entirely, or at least until a very wide range of upgradeable GPUs are available. The RAM is also going to be costly, given the few slots available and the higher price for denser modules. I love the design, but this is the wrong class of computer for the constraints that come from the execution of that design.