- Amazon keeps at least 3 copies of your data (which is what you need for high reliability) in at least 2 different geographical locations. That's what we'd do ourselves, too, if we continued to use our own storage internally. So your math is off both on the storage costs and then the costs of maintaing two or more datacenters and the networks between them.
- When Amazon reduces their prices, you instantly get all your storage cheaper. This isn't something you get with your capital expenditure of disks - your costs are always fixed. This has upsides and downsides, but you certainly don't get instant price breaks to your OpEx costs. When they added cheaper, tiered storage, our bill with Amazon dropped hugely.
- There's built-in price pressure with Amazon, too. The cost of one month's rent is roughly the same as the cost of leaving. So if it gets too expensive (or unreliable or slow or whatever your metrics are), you can easily leave. And Amazon has incentive to keep lowering prices and improving speed & reliability to ensure you don't leave.
- CapEx sucks. It's hard on your cashflow, it's hard on your debt position if you need to lease or finance (we don't, but that just means it's even harder on our cashflow), it's hard on taxes (amortization sucks), etc etc. I vastly prefer reasonable OpEx costs, with no debt load, which is what Amazon gets us.
- Free data transfer in/out of EC2 can be a big win, too. It is for us, anyway.
- Our biggest win is simply that it's easy. We have a simpler architecture, a lot less people, and a lot less worry. We get to focus on our product (sharing photos) rather than the necessary evils of doing so (managing storage). We have two ops guys for a Top 500 website with over a petabyte of storage. That's pretty awesome.
Hope that helps!