Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why We Moved Off The Cloud

code.mixpanel.com

Post reply on HN
Print view View original

11–20 of 154 posts

Re: Why We Moved Off The Cloud

#11

Post by powertower »

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much as their cloud equivalents and we're not even co-locating.

I've been saying this for ages, and every time people would fall over backwards trying to defend/prove their cloud mistake...

"The cloud is cheaper, faster, and infinitely scalable."

Except none of those 3 is true for any real world use case, but a few.

The moment a popular site like Reddit switches to the cloud, is the moment it becomes barely usable during certain times of the day.

Re: Why We Moved Off The Cloud

#12

Post by matwood »

bryanh wrote: post #2

Just about everyone and their mom hopped onto the cloud bandwagon there for good time, but with this steady onslaught of praise for bare-metal hosting, maybe this will reverse a little. It seems like the only thing cloud really provides best is for: 1) Short lived instances or "now" instances. 2) and... what? I'm trying to think of other situations where cloud beats bare metal, but I am coming up short.

I think the cloud is also good for getting an MVP out the door. The cloud makes sense for startup that has few employees and wants to focus on the product rather than managing bare metal. Of course if/when they become successful it makes sense to move onto their own infrastructure.

This how I always thought it went, so stories of growing companies moving off the cloud don't seem like a big deal - just a natural progression.

Re: Why We Moved Off The Cloud

#14

Post by pablasso »

powertower wrote: post #11

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much as their cloud equivalents and we're not even co-locating. I've been saying this for ages, and every time people would fall over backwards trying to defend/prove their cloud mistake... "The cloud is cheaper, faster, and infinitely scalable." Except none of thos…

I don't think people argue solely on hardware specs, you should count the maintenance and related salaries into the total cost.

Re: Why We Moved Off The Cloud

#16

Post by larrys »

"You would think that you could get better prices by signing 1 or 2 year contracts, but interestingly enough, out of the initial 5 providers we talked to the two that didn’t require contracts had the best prices." (snip) "We’ve moved 100% of our machines that rely upon performant (sic) disks to dedicated servers hosted at Softlayer. Roughly speaking, this corresponds to about 80% of our hosting costs. Eventually, we’ll move everything "

If you don't have a contract there is nothing to prevent a provider from raising prices on you. The reason to have a contract is not just to get the best price. It's to have a price guarantee. Edit: Moving 100% of your machines won't be something you will want to do. If you have a contract you can renegotiate well in advance of any price increase.

Prices always drop?

People thought housing prices always go up as well.

How long is the price guaranteed for? I'm assuming mixpanel has this issued covered but it's important to keep it in mind. Not having a contract goes both ways.

Re: Why We Moved Off The Cloud

#17

Post by powertower »

pablasso wrote: post #14
powertower wrote: post #11

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much as their cloud equivalents and we're not even co-locating. I've been saying this for ages, and every time people would fall over backwards trying to defend/prove their cloud mistake... "The cloud is cheaper, faster, and infinitely scalable." Except none of thos…

I don't think people argue solely on hardware specs, you should count the maintenance and related salaries into the total cost.

The majority of that market is getting VPSs or renting dedicated boxes. There is no hardware maintenance or related salaries.

Even for multiple boxes, you call up your provider, and order a cage with hardware filled. They'll provide everything.

95% of "Cloud" is just a new marketing term for VPS.

Re: Why We Moved Off The Cloud

#18

Post by MichaelApproved »

"We recently added a new backup machine with a crappy CPU, little RAM, and 24 2TB drives in a hardware RAID 6 configuration. You can’t get that from a cloud provider and if you find something similar it’s going to cost an order of magnitude more than what we’re paying."

I tend to agree with his points but, for back ups the cloud is perfect. If he stayed in the cloud, he wouldn't even need the server in question.

Re: Why We Moved Off The Cloud

#19

Post by bitemyapp »

powertower wrote: post #11

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much as their cloud equivalents and we're not even co-locating. I've been saying this for ages, and every time people would fall over backwards trying to defend/prove their cloud mistake... "The cloud is cheaper, faster, and infinitely scalable." Except none of thos…

I agree with everything you said except for:

>The moment a popular site like Reddit switches to the cloud, is the moment it becomes barely usable during certain times of the day.

Reddit is largely on EC2, do you even know what you're talking about?

EC2 is part of Reddit has so many problems, among others.

Re: Why We Moved Off The Cloud

#20

Post by ChuckMcM »

powertower wrote: post #11

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much as their cloud equivalents and we're not even co-locating. I've been saying this for ages, and every time people would fall over backwards trying to defend/prove their cloud mistake... "The cloud is cheaper, faster, and infinitely scalable." Except none of thos…

So I'm running operations at Blekko, and just prior to that I was at Google in their eng/ops organization. I had been doing a whole lot of 'total cost of ownership' aka TCO computations around engineering infrastructure both for Google and of course now for Blekko.

The conclusion I came to is that for a 'web 2.0' type setup, the break even point was about 500 'machines.' That was in part because a 'machine' today has 8 - 24 'threads' and 2 - 40T of 'storage' and (at the time) 2 - 96G 'memory.' So in terms of 'cloud' you could easy run 10 "instances" on these sorts of machine. So 500 machines might be 5000 'instances' in an AWS type cloud.

Its this '10:1' multiplier effect (which is only getting better with bigger machines) and the management techniques of running the same config everywhere, etc. Means your TCO goes up more slowly than the capacity of the resulting infrastructure, so you can 'solve for x' where the two lines cross to identify the break even point. Everything east of that point you're coming out ahead of a 'cloud' based deployment.

What is still a challenge however is geographic diversity. If you wanted to put 500 machines 'around the world' so 125 machines in each 90 degrees (approximately) of longitude, the economics of getting 5 - 10 'cabinets' in places around the world can work against you. (you have more negotiating power if you're putting in 100 racks than if you are putting in 10 racks)

Post reply on HN