Ask HN: AWS or dedicated server?

EBS has great performance. RightScale noted that they got over 70MB/s with sysbench and over 1000 I/O operations per second. If you want more performance, you can even stripe across EBS volumes.

EBS really helped EC2's viability a ton. EC2 users now have access to cheap, reliable, and fast storage.

In terms of the processor metric, that's harder to gauge. Right now, Amazon says one EC2 compute unit is roughly equivalent to a 1.0-1.2GHz 2007 Xenon or Opteron (or a 1.7GHz 2006 Xenon which was their original documentation).

Think of it this way, Amazon is putting you on a beefy server with some other people. I'd guess these servers are 4-core boxes running at around 2GHz+ with 16GB of RAM. So, with the Extra-Large instance, you're basically getting the box (15GB of RAM, 4 cores with 2 compute units per core (roughly 2-2.4GHz per core)). So, with the Large instance, you're getting half of the server (2x 2GHz Xenon processors) and with the small instance you're probably getting one core at half speed.

And that's really as much as most people will need especially since I'm guessing there's a bit of bursting ability to the CPU capacity.

Hope that helps make Amazon's CPU situation a little more understandable.

So the main purposes are simple horizontal scaling and efficient use of hardware. Virtualization makes horizontal scaling simple because it's just a matter of cloning a particular machine (or machines). It makes efficient use of hardware because there's no need to have a dozen physical machines for a dozen different purposes, unless they all use a full machine worth of resources.

Lets say I want to add a new (non-static) Web server. Well, Apache is on its own VM; I can clone and migrate it to a new physical box (or just have two on the same hardware). I could also simply add more resources dynamically. If I need to scale the database, same deal. The biggest win here is that when I scale one of those, nothing else comes with it. There's no DNS server on the Web box. There is no NFS server on the database box.

Before virtualization, you basically had two choices: Throw a whole bunch of packages on a single box or spread it out over different physical machines. The first completely ruins encapsulation, thus adding unnecessary complexity, while the second is really uneconomical unless you're using all those resources out of the gate.

Then, what happens when you need to scale? All that crap needs to be setup again! Hopefully we were smart about it and made it as simple as possible, but I have never made a system as scalable as my current one-click-cloning mechanism.

Right now we only have a single physical machine (16 cores, 32gb ram, iSCSI); using your recommendation of "whatever packages on whatever servers" I would end up with this clusterfuck of a server that does a dozen different things at once. What I have now is exactly that, except encapsulated into VMs with their own resources and their own purpose.

Yes, virtualization has a slight performance cost (though bare-metal virts like Xen have a pretty marginal one), but I'll gladly accept it for the massively easier scaling and efficient use of hardware. And, yes, if one VM happens to go insane for some reason, it doesn't affect anything else. For instance, on one of our older servers, the MySQL VM's "drive" had a tendency to become corrupt randomly. I never really did figure out why, but I imagine it was because I make fun of MySQL all the time, but I digress -- the point is, it never affected the rest of the machine and, since MySQL was used for things of little importance (Wordpress), it didn't even take down the website when it happened.

That was a pretty rambling explanation, but hopefully it covers your questions. If not, let me know.

There is the potential that you don't want to delve into what you're paying for stuff too much, but it just seems like AWS is charging similar rates to ThePlanet and SoftLayer which are the two dedicated hosts that seem to have the most credibility in the community. Even if you were provisioning your own 1.7GB instances on a larger dedicated box, you would still only fit about 8 or 9 of them in 16GB of RAM (leaving room for Xen and such) which would make it the same price as AWS. The only thing I can see is that the included bandwidth could save some money. Maybe I'm not good at looking for dedicated server deals.

- Depending on your storage requirements, 2 drives + RAID 1 (which is more of a convenience than anything and I almost never recommend getting) is often times more expensive than an iSCSI LUN which is far superior and offers zero-setup cross-country replication and snapshots (if we're going to pretend that RAID 1 is some kind of backup solution).

- When ordering, if you choose the lowest clock speed CPUs, you're practically guaranteed to get the highest rated (more expensive ones) for free. This is either due to scarcity of low-end CPUs or Softlayer loves me. I have ordered numerous boxes from them and this has always been the case.

- They always have "specials" which are usually pretty ridiculous. For instance, 16 of the 32GB of RAM we have was free, as in beer. Right now (and most of the time) they have free double RAM and HDD. Kiss the cost of one of those RAID drives away.

- There are non-monetary considerations, such as support. Softlayer has without a doubt the best technical support I have ever been provided, and I've been through countless hosts in my tenure. We're talking about an Unmanaged host that has better techs than any Managed host I've come across. Not to mention conveniences such as automated OS reloads, private network, inter-DC OC3 backbones, VPN, secure backups, optional CDN, etc. (AWS has most of these afaik, minus VPN, but this goes to equivalence)

- Your 4-core server, if you don't make use of deals, would likely be equivalent to AWS. Once you start getting into high core-counts, that changes fast. As a huge proponent of parallelization, many of the processes run for TicketStumbler make use of multiple CPUs; this means a lot of what we do is CPU-bound, thus the need for higher core counts.

- 2TB of bandwidth is included; I also have no idea how this affects the cost overall. Edit: I added a couple TB of transfer to the AWS calculator, plus 80GB of storage: $854.10 per Extra Large. The difference in cost between this and our machine now amounts to nearly nothing.

So, at the end of the day, the hardware we have is nearly identical in cost (within $100, IIRC) to the Extra Large Linux instance you reported, while having twice the number of CPU cores and twice the amount of RAM. We're also afforded all the other luxuries that come with the myriad services and support the conventional dedicated host provides.

The dedicated hosting environment also allows me to setup and administer the hardware in the method I described in my previous reply; i.e., I don't have to setup a single Extra Large Instance (well, technically two) to handle a dozen different jobs.

Hope this helps! Let me know if you have any other questions.