I did high-security systems so let's modify that for what I would trust. Five servers that offload TCP/IP, a firewall, VPN, and packet/session authentication onto a PCI card with hardened RTOS (ex: Sentinel's HYDRA firewall). Let's me send sanitized data directly to application through trusted interface among other things. Further, the business's I.P. must be protected: physical separation rather than virtualization + legal protection through regulation and contracts. It will also need to run constantly, be tested periodically by intentional fail-overs, and have predictable cost despite this.
So, I spend 10-30 minutes Googling. I find Hong Kong and Switzerland are among best for regulations on data. Settle on Switzerland for various reasons. I haggle for a few hours with hosting companies to set up the deal. I have some hardware shipped 1-3 business days from other companies. We spend a few hours setting it up, doing disk encryption, configuring the guard (PCI card), setting up link encryption, and testing both sides in various ways. We save & standardize anything we figure out for scripted, instant deployment and testing when we scale out. I ship the relevant boxes to the datacenter. Meanwhile, I deploy the local boxes and thoroughly test the site.
Offshore site is probably up before the week is over. I spend a day re-running the tests. I simulate a bunch of fail-overs in various situations to make sure it works no matter what. Once I'm confident in it, we move all the relevant data into both of them. We might have already done that before shipping it if the situation allowed & then we just move what changed. We transition our domain to point at the new service. The users use it. If it fails, it fails over to the other one. The cost is likely $1,000+/mo and pretty flat except for the times where it takes the main load. Not likely to overdo my cap with only 3 webservers, though.
So, in summary, I have several secure boxes, total control of them, data at rest protection, data in transit protection, regulatory protection, acceptable network speed, predictable albeit higher cost, and long-term stability in various ways. It took around two weeks last time I did something like this. Given they usually charge for the rack, best to always ship a few extra boxes that stay off or idle: reduces impact of shipping time when boxes fail or workload increases.
What does Amazon charge and with what wait for a comparable offering with strong host, network, and I.P. protection?