Earlier quoted context omitted.
OP. I tried to address that in the post. I'm not sure how to explain it differently. The only reason you need to spin up more instances to handle the load is because of how slow the machines are in the first place. That $750 you spend on 2x m3-xlarges could get you machines that can handle 4x the load (or more). Under normal conditions, you spend $750/m. Under peak, you'll spend $3000/month with EC2 but still $750/m…
The main issue with the hybrid approach is devops cost. you will be basically dealing with two very different infrastructures, and building the ops for such a hybrid system could be costly (in money and time). Taking this into account, the money you save shrinks, In addition, a more complex system most likely has more parts to fail as well, so your total reliability might suffer somewhat as well. Another thing to tak…
In terms of inter-datacenter network reliability: You have to deal with this if you want reliable hosting anyway. If you spread your database and app servers across data centers, yes, you are begging for problems and the problem is that your app is not designed for resilience.
But you can easily enough do "hybrid" within the same datacenter, if you opt for any of the number of EC2 alternatives from companies that also do dedicated hosting.