Live data from Hacker News

How and Why Swiftype Moved from EC2 to Real Hardware

highscalability.com

21–30 of 194 posts

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#21

One question. If real hardware was always 50% cheaper then AWS wouldn't have been such a success. Can you please explain in which scenarios it makes economical sense to use AWS compared to real hardware?

Real hardware has always generally been cheaper.

AWS is a success because there are no upfront costs, it lets you scale up very quickly, and you don't need in-house hardware expertise to maintain your machines. People are willing to pay a premium for these advantages.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#22

This is a no-brainer if you've ever done anything at scale. The explanation is rather simple - hardware is always "on the premise", yours or Amazon's. Someone needs to swap drives, motherboards, man the networking gear, run cables, etc. Amazon doesn't really get a break on the hardware cost because 10,000 servers do not cost less per server than 100 servers (in fact they cost more as the volume goes up if you need th…

Why would 10,000 servers cost more than 100 servers? It seems like if you are buying all of your parts in bulk they are going to be cheaper. I'm pretty sure Intel's pricing on CPUs are cheaper by the tray than individually.

I know at least when I've bought 20-30 servers at a time, I was able to get a lower cost than when I've only been buying one.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#23
post #18

One question. If real hardware was always 50% cheaper then AWS wouldn't have been such a success. Can you please explain in which scenarios it makes economical sense to use AWS compared to real hardware?

in which scenarios it makes economical sense to use AWS compared to real hardware? There's three types of workloads that make sense to run on EC2: a) Extremely spiky/seasonal loads (batch jobs, event/campaign traffic) b) Loads that can be structured as to run entirely from spot-instances (worker-pools) c) Loads so small that the markup versus rented/dedicated hardware just doesn't matter

Thanks for the answer. Maybe I need to understand better the advantages and disadvantages of cloud.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#24
post #16

Earlier quoted context omitted.

AWS is always going to include a premium because they take care of the DevOps portion of your infrastructure. There are plenty of virtual hosting companies that cost significantly less than dedicated hardware, if you won't need all the bells and whistles.

> AWS is always going to include a premium because they take care of the DevOps portion of your infrastructure. But wouldn't that apply also to SoftLayer?

No, there is significantly more complexity, overhead, and R&D to providing cloud services in comparison to bare metal. SoftLayer is actually a very expensive bare metal server provider. There are several good options that cost less than 1/3rd the price. Realistically, at just modest scale (a few physical servers), you should see 1/6th the cost of Amazon.

The main benefits of Amazon is that it: a) allows you to scale down i.e. buy services in smaller portions than complete physical servers and b) APIs c) integrated features

You could probably pay for one devops position once your infrastructure gets to 10 physical servers.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#25

One question. If real hardware was always 50% cheaper then AWS wouldn't have been such a success. Can you please explain in which scenarios it makes economical sense to use AWS compared to real hardware?

In my personal opinion (based on some real experience) EC2 for a startup makes a lot of sense for prototyping your application and your infrastructure (when you don't really know what is it you're building and what components you're building it from). At this stage you just is it as an easy way to get a set of Linux computers connected to the Internet.

When you get to a point where you feel like this whole thing is going to fly, I'd recommend starting to think if paying the "cloud tax" (resources spent around EC2 stability issues and the cloud-specific stuff) a good idea in your particular case. There are some companies that benefit greatly from the elasticity of the cloud (the ability to scale up and down along with their specific load demands), but many companies aren't like that. If your traffic is relatively stable and predictable (you do not have 10-100x traffic surges) and your infrastructure load does not grow linearly with the traffic, using real hardware over-provisioned to handle 2-5-10x traffic spikes without huge decrease in performance may be a better idea in terms of the cost.

Of course, you could start the company based on all of the PaaS magic sauce (databases, queues, caches, etc) provided by Amazon nowadays and only use EC2 to run your application code (AFAIU that's the ideal use-case for AWS) and just kill misbehaving nodes when an issue occurs, but then you need to factor AWS costs into your business plan because migrating away from a PaaS is almost impossible at any large scale, so you are going to stay with Amazon for a very long time.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#26
It strikes me that more and more a critical selection when growing in this way is the DNS part. It needs to be back-end agnostic and provide an increasing amount of functionality.

Health checks and failover are must have now, but this article makes me wonder three things:

1) Are there any DNS services that understand geography of your "zones", i.e. route to and failover based on IP? (but are still platform agnostic).

2) How long can a DNS failover take worst case? You can technically set a low TTL, but don't a lot of ISPs just increase that to a minimum?

3) Isn't it better to replace some of the DNS failover with high availability dedicated load balancing?

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#27

This is a no-brainer if you've ever done anything at scale. The explanation is rather simple - hardware is always "on the premise", yours or Amazon's. Someone needs to swap drives, motherboards, man the networking gear, run cables, etc. Amazon doesn't really get a break on the hardware cost because 10,000 servers do not cost less per server than 100 servers (in fact they cost more as the volume goes up if you need th…

> The explanation is rather simple - hardware is always "on the premise", yours or Amazon's. Someone needs to swap drives, motherboards, man the networking gear, run cables, etc.

> So you're paying Amazon to do the same work you would do otherwise - only you're subject to their rules and procedures and Amazon being a profitable business needs to mark their services up.

But I thought that they were paying Softlayer to do that stuff instead of Amazon. They're not doing it themselves - and yet it's still cheaper!

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#28

One question. If real hardware was always 50% cheaper then AWS wouldn't have been such a success. Can you please explain in which scenarios it makes economical sense to use AWS compared to real hardware?

Well for me personally it makes sense if you are not of very large scale. Real hardware is cheaper but the costs of electric, transit, and colo space are not your only costs. If you are going to do physical hosting right you need to be in several locations, and you need to have a very good idea of what assets you will need for a period of time.

With AWS I can scale up easy, not have to worry about doing things like replacing failed hard disks, and most importantly I can be in multiple geographic sites for no additional cost. That to me right now is worth a 50% premium as the cost for doing that would be higher than that savings.

I think if you reach a certain scale, and have predictable usage, it is not a bad thing to setting up cabinets in 2 or 3 locations. We have found too that a lot of Colos are getting bought up and then will not lease you a few cabinets. They want to sell only to people who want a cage, or entire room. It is hard for small to medium sized businesses.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#29
post #18

One question. If real hardware was always 50% cheaper then AWS wouldn't have been such a success. Can you please explain in which scenarios it makes economical sense to use AWS compared to real hardware?

in which scenarios it makes economical sense to use AWS compared to real hardware? There's three types of workloads that make sense to run on EC2: a) Extremely spiky/seasonal loads (batch jobs, event/campaign traffic) b) Loads that can be structured as to run entirely from spot-instances (worker-pools) c) Loads so small that the markup versus rented/dedicated hardware just doesn't matter

Yes, completely agree.

Maybe I'd just add one more case here: some users are OK with locking themselves up to AWS by treating it as a platform from the day one and building on top of AWS database/queue/etc services. For those people using EC2 just to run the app code and replacing instances when they misbehave may be a good idea.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#30

One question. If real hardware was always 50% cheaper then AWS wouldn't have been such a success. Can you please explain in which scenarios it makes economical sense to use AWS compared to real hardware?

AWS is always going to include a premium because they take care of the DevOps portion of your infrastructure. There are plenty of virtual hosting companies that cost significantly less than dedicated hardware, if you won't need all the bells and whistles.

> AWS is always going to include a premium because they take care of the DevOps portion of your infrastructure.

This is a surprise to me, given that I work at an AWS shop doing things other people would call "DevOps". AWS doesn't automate provisioning or provide a (worthwhile) deployment pipeline, andAWS doesn't react (except in crude and fairly stupid ways) when something goes wrong or out-of-band.

Post reply on HN