Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why We Moved Off The Cloud

code.mixpanel.com

Post reply on HN
Print view View original

91–100 of 154 posts

Re: Why We Moved Off The Cloud

#91

Post by rachelbythebay »

ChuckMcM wrote: post #20
powertower wrote: post #11

> One last thing about getting dedicated hardware. It's cheaper a lot cheaper. We have machines that give us 2-4x performance that cost less than half as much as their cloud equivalents and we're not even co-locating. I've been saying this for ages, and every time people would fall over backwards trying to defend/prove their cloud mistake... "The cloud is cheaper, faster, and infinitely scalable." Except none of thos…

So I'm running operations at Blekko, and just prior to that I was at Google in their eng/ops organization. I had been doing a whole lot of 'total cost of ownership' aka TCO computations around engineering infrastructure both for Google and of course now for Blekko. The conclusion I came to is that for a 'web 2.0' type setup, the break even point was about 500 'machines.' That was in part because a 'machine' today has…

Beware the "same config everywhere" approach. It works up to a point, then it turns into a disaster. All you need is one totally broken change like "chmod -x /usr" to really make life interesting. You start bleeding machines and pretty soon you have nowhere left to host your tasks.

It's interesting, right? At first, you can handle a couple of totally mixed-up machines. Then it stops scaling and you have to start doing the whole "golden + syncer" approach.

Then you go too far and get into a monoculture. When the machines do break, it's impossible for humans to go around and fix them in any reasonable amount of time because there are too many. It's amusing when this happens and the solution put forth is "more administrative controls".

Re: Why We Moved Off The Cloud

#92

Post by joevandyk »

One thing I love about ec2 is that I can easily test/debug/modify my provisioning recipes (written in chef) from a blank slate.

With fog (http://fog.io/1.0.0/index.html), I can start up a new ec2 instance in less than a minute and tell it to run the chef process. If it doesn't work properly, I shut it down and try it again.

How does that work on a dedicated machine at, say, SoftLayer?

Re: Why We Moved Off The Cloud

#93

Post by code_duck »

Yes, my experience with the problems of performance on virtual servers is disk related. That's great that you get guaranteed CPU, memory, bandwidth etc... if you're getting 3 MB/s disk throughput, it doesn't matter, your site will slow to a crawl. I moved away from Slicehost for this reason, and have never had such issues with Linode.

Re: Why We Moved Off The Cloud

#94

Post by jules »

papercruncher wrote: post #49
jedberg wrote: post #48

You didn't abandon "the cloud", you just switched providers. You're still paying someone else for servers that you don't own (unless softlayer ships you those machines after 3 years). This is why I hate the term "the cloud" -- because it is too nebulous and non-descriptive.

Cloud implies running in virtualized environment. Dedicated implies that only your bits run on that hardware which is huge for I/O.

Is that because multiple instances will be running with the same disk? As far as I know virtualization doesn't have a big overhead in itself, or does it?

Re: Why We Moved Off The Cloud

#95

Post by TimH »

jedberg wrote: post #50
papercruncher wrote: post #49

Earlier quoted context omitted.

Cloud implies running in virtualized environment. Dedicated implies that only your bits run on that hardware which is huge for I/O.

> Cloud implies running in virtualized environment. Dedicated implies that only your bits run on that hardware which is huge for I/O. True, but the blog post didn't say if they are using virtualization or not.

So they may or may not have abandoned the cloud?

Re: Why We Moved Off The Cloud

#96

Post by jread »

jedberg wrote: post #48

You didn't abandon "the cloud", you just switched providers. You're still paying someone else for servers that you don't own (unless softlayer ships you those machines after 3 years). This is why I hate the term "the cloud" -- because it is too nebulous and non-descriptive.

Infrastructure as a service (IaaS) cloud generally implies a few distinguishing features such as hourly utility billing and no long term commitments, rapid elasticity (provision/de-provision in minutes, not hours) and usually (but not always - there are bare metal clouds) virtualization. People often refer to software as a service (SaaS) as cloud, which I think is where it starts getting very ambiguous (i.e. is any hosted web application in the cloud???)

Re: Why We Moved Off The Cloud

#97

Post by powertower »

jwegan wrote: post #63

Rackspace's primary business is dedicated hosting. In fact they let you have both dedicated hosts and cloud hosts and allow dedicated and cloud hosts to talk to each other. I'm curious why they didn't just switch to Rackspace's dedicated hosting. It would have given them the performance they needed while retaining the flexibility of being able to quickly spin up cloud machines in the same datacenter as the dedicated…

> In fact they let you have both dedicated hosts and cloud hosts and allow dedicated and cloud hosts to talk to each other.

That's not amazing at all.

You do realise that a "cloud" host and a dedicated box are two exact same hardware boxes siting next to each other in a rack? One's just virtualized 10x with Xen, VMWare, KVN, etc.

Re: Why We Moved Off The Cloud

#98

Post by powertower »

AdamGibbins wrote: post #87
kqueue wrote: post #72

This service provider has great pricing compared to softlayer http://www.hetzner.de/en/hosting/produktmatrix/rootserver-pr... Intel Corei7-2600 Quad core + 16GB DDR3 + 2 x 3TB 7200 for 49 euro.

The quality of service and network is also widely different and varied. Hetzner are awesome for the cost, don't get me wrong. But at times their network is terrible and their service seems to vary hugely. You don't get these problems with Softlayer, you pay significantly more and get significantly better almost-guaranteed service.

If you're looking for tier-1 network traffic, 100tb.com is with Softlayer, and since they have so many servers, they get massive discounts and can give you 100TB of softlayer traffic for relativly cheap.

Re: Why We Moved Off The Cloud

#99

Post by whalesalad »

larrys wrote: post #89
jedberg wrote: post #48

You didn't abandon "the cloud", you just switched providers. You're still paying someone else for servers that you don't own (unless softlayer ships you those machines after 3 years). This is why I hate the term "the cloud" -- because it is too nebulous and non-descriptive.

"it is too nebulous and non-descriptive" Agree but wouldn't you say that sole use of physical hardware (whether or not you own or lease that hardware and regardless of physical location) is really the issue that separates cloud from non cloud? Example: We have servers that we own in our offices. We have servers that we own in colocation. We have some dedicated virtual servers at MT http://mediatemple.net/webhosting/d…

I wouldn't consider the (mt) servers 'cloud', but rather just as you said it, shared. I too hate the cloud term, for reasons mentioned above (by jread).

Re: Why We Moved Off The Cloud

#100

Post by dholowiski »

jwegan wrote: post #63

Rackspace's primary business is dedicated hosting. In fact they let you have both dedicated hosts and cloud hosts and allow dedicated and cloud hosts to talk to each other. I'm curious why they didn't just switch to Rackspace's dedicated hosting. It would have given them the performance they needed while retaining the flexibility of being able to quickly spin up cloud machines in the same datacenter as the dedicated…

Rackspace is very expensive... almost 2x their competitors. At work we left rackspace recently, because we needed to upgrade our server and it was going to jump up to $1200! a month. Plus we found out that we were running on two year old discontinued hardware. Although we still had a SLA that didn't apply to hardware failures, which would be fixed on a 'best effort' basis and could mean a days-long outage. We moved (to the cloud ironically) and got a huge performance increase, for half the price (and that includes paying a company to monitor/manage it for us).
Post reply on HN