Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How and Why Swiftype Moved from EC2 to Real Hardware

highscalability.com

Post reply on HN
Print view View original

61–70 of 194 posts

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#61

Post by rodgerd »

snoopybbt wrote: post #43

There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing: Cloud computing offers you the great and awesome advantages of being able to instantly scale your application, replicate your data and basically just grow according to your business volume, and all this without significant investments, delivery time, setup time, people time, maintenance or anything bu…

> There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing:

I wish it were true, but plenty of companies are gripped by cloud fever. I've seen quite a few going down the route of charging into the cloud not because they've run the numbers and found it stacks up, but because they want to be in the cloud, and Amazon have some great marketing people.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#63

Post by moe »

jarjoura wrote: post #13
nanoGeek wrote: post #10

One question. If real hardware was always 50% cheaper then AWS wouldn't have been such a success. Can you please explain in which scenarios it makes economical sense to use AWS compared to real hardware?

AWS is always going to include a premium because they take care of the DevOps portion of your infrastructure. There are plenty of virtual hosting companies that cost significantly less than dedicated hardware, if you won't need all the bells and whistles.

because they take care of the DevOps portion of your infrastructure

Sorry, but that is mostly a lie.

Running a non-trivial app on EC2 is significantly more complex than doing the same on (rented) bare metal. Scaling to a massive size can be easier on EC2, but only after you paid a significant upfront cost in terms of dollars and development complexity.

Is your app prepared to deal with spontaneous instance hangs, (drastic) temporary instance slowdowns, sudden instance or network failures?

Did you know that ELBs can only scale up by a certain, sparsely documented amount per hour?

Or that you need a process to deal with "Zombie" instances that got stuck while being added/removed to ELBs (e.g. the health-check never succeeds).

Or that average uptime (between forced reboots) for EC2 instances is measured in months, for physical servers in years?

Or that Autoscaling Groups with Spot instances can run out of instances even if your bid amount is higher than the current price in all but one of the availability zones that it spans?

The list of counter-intuitive gotchas grows very long very quickly once you move an EC2 app to production.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#64

Post by zymhan »

lclarkmichalek wrote: post #49
nanoGeek wrote: post #31

Earlier quoted context omitted.

Great point. Also, on AWS if you use Amazon Linux you are even more 'locked-in'. Maybe that's the reason many hosting companies give you discounts at the beginning.

Pretty sure Amazon Linux is just their particular flavor of RHEL, right?

Correct. It doesn't have the exact same default configuration as, say, CentOS, but it's not much different otherwise.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#66

Post by kovyrin »

snoopybbt wrote: post #43

There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing: Cloud computing offers you the great and awesome advantages of being able to instantly scale your application, replicate your data and basically just grow according to your business volume, and all this without significant investments, delivery time, setup time, people time, maintenance or anything bu…

If only it were that simple... But every second relatively young engineer I interview points out that "But Netflix! Look how good the cloud works for them!" and once again I need to explain, that Netflix spends millions in engineering resources cost to handle EC2 issues and we are not (almost nobody is) Netflix (yet?).

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#67

Post by kovyrin »

bobofettfett wrote: post #33

My 5cent: If you have lots and lots of money and a high margin business, do yourself a favor and go with Amazon (much less hassle with contract management and low level challenges). If you need to scale month to month and are growing 50% per month, go with Amazon. If you are very small and can live with 10 instances, go with Amazon. If CAPEX doesn't help you and for whatever reasons you need to spend OPEX, go with Am…

Absolutely, 100% agree. Thanks for the comment!

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#69

Post by skwirl »

snoopybbt wrote: post #43

There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing: Cloud computing offers you the great and awesome advantages of being able to instantly scale your application, replicate your data and basically just grow according to your business volume, and all this without significant investments, delivery time, setup time, people time, maintenance or anything bu…

>you're also paying Amazon's profits.

Are you? Amazon is not known for posting profits, and is quite possibly operating AWS at a loss.

They have promised to show financial results for AWS itself with the quarterly report being released next month, so we'll see.

Re: How and Why Swiftype Moved from EC2 to Real Hardware

#70

Post by yeukhon »

rodgerd wrote: post #61
snoopybbt wrote: post #43

There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing: Cloud computing offers you the great and awesome advantages of being able to instantly scale your application, replicate your data and basically just grow according to your business volume, and all this without significant investments, delivery time, setup time, people time, maintenance or anything bu…

> There's not much need for a fancy article on a fancy website in order to understand a key concept of cloud computing: I wish it were true, but plenty of companies are gripped by cloud fever. I've seen quite a few going down the route of charging into the cloud not because they've run the numbers and found it stacks up, but because they want to be in the cloud, and Amazon have some great marketing people.

From personal experience this is absolutely true. The initial cost to migrate and learn the cloud practice is unbelievable, but once you have the process in place running on Amazon CAN save some cost down the road, including the hours you need to replace hardware. You will build tools or use existing tools to create your infrastructure and operation process. A generous estimation to reach a level of maturity is 1.5 year. For ever-growing business this cloud fever is acceptable.

You can definitely save cost by subscribing to reserved instances, but the downside is you have to put down money upfront, which is very hard for many small players out there.

But watch out if you run data pipeline jobs - sometimes your so-called big data is really not that big. A few GBs daily report doesn't need to run on c3.xlarge instances. They can do just fine with a 24/7 m3.large instance. There was an article on HN a while ago about how one could run a custom report with shell commands on a commodity hardware, and get 100x times performance compare to running on EMR. You can also consider running most of your jobs on premise. Ihe network banwidth in/out is probably going to be cheaper than running all of your jobs on EMR. Direct connect is a great choice to boost the connectivity stability and security. Go for it.

Cloud is great for HA, because on Amazon you are encourage to build in multi-AZ and even multi-region. S3 is absolutely the de-facto today IMO for object storage. It's cheap and reliable. The learning curve for proper Amazon (or just about any cloud provider) is really deep. You can either end up like running Black Friday sales, or running like Netflix with monkey enjoying tea.

Running on cloud is no different than running on-premise, just you have to start all over again, because now you have to re-consider network, security, monitoring, and practice.

Post reply on HN