Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we are not leaving the cloud

about.gitlab.com

Post reply on HN
Print view View original

141–150 of 216 posts

Re: Why we are not leaving the cloud

#141

Post by tetha »

drdaeman wrote: post #122
tetha wrote: post #82

Earlier quoted context omitted.

My company has done some research on that, in the webscale bubble. It is wrong to think of the divide as "Cloud" vs "Bare Metal" in a VM for server thing. Disregarding up-front investment, you're comparing rented VMs in the cloud with bare metal boxes, datacenter costs, and most important, manpower. At a small scale, you're paying the cloud less than 2 or 3 competent admins with datacenter and networking experience c…

Cloud may save from hardware/driver issues - by providing an already tested environment, that doesn't (normally) trip on some weirdness in, say, network drivers. But if the problem is in the software stack, one would still need competent sysadmins/system engineers with skills to diagnose and resolve the issue. When (just a random example) oom-killer wreaks havoc and free(1) insists there's more than half of physical…

This is why I said hardware op.

If you use some VM-based cloud infrastructure you will still need a bunch of good system/linux admins to make your application work on the linux in the cloud. Without these, you're toast.

Bare metal requires you to have both good system and linux admins, and on top of that, good hardware admins, networks and datacenter hands. These are different skill sets.

So again: If you do the bare metal thing right at the scale it pays for itself, you'll probably end up paying both the skillset for the cloud deployment, and add the hardware skillset on top.

Re: Why we are not leaving the cloud

#142

Post by majewsky »

user5994461 wrote: post #125
dsr_ wrote: post #67

Earlier quoted context omitted.

Will you stop calling me a liar if I correct that from "the costs" to "your costs"? I am assuming that you are growing, and I think there's a provable assumption that for high levels of service, hiring someone else to do it is more expensive than in-house.

> hiring someone else to do it is more expensive than in-house. Except you're gonna need between 10 and 100 people to recreate the service in-house, not just one. Trying to re-do something from scratch in-house is always more expensive than just buying the product.

I get the sentiment, but it really depends: For some services, it's very true. For other services, careful problem analysis will show that you only need one or two features from the full package, which then can be replaced by a very small shell script.

Re: Why we are not leaving the cloud

#143

Post by hueving »

amorphid wrote: post #81
tbrowbdidnso wrote: post #34

So everyone says the cloud is the future. I get it. But is this the truth, or what all the tech giants want people to believe? Don your foil hats for a moment and listen to me. All the original internet companies run their own hardware. They rent out excess production capacity to us peons in the form of cloud services. These companies that all run their own hardware exclusively are telling everyone that it's stupid t…

I don't know that I'd say Netflix is completely at the mercy of Amazon. Sure, if AWS goes down, Netflix goes down. But that would be true enough for they ran all of their own stuff, too... "Netflix is completely at the mercy of their ops team". One of the big points of the cloud is that is gives one the ability to do things that one would trouble doing without the cloud. A small business with a single part-time devel…

>But that would be true enough for they ran all of their own stuff, too... "Netflix is completely at the mercy of their ops team".

They employ their ops team and their ops team has equity stake in the company's success as well as their career riding on the line.

If a chunk of Amazon goes down and takes out your website, all you can do is wait and hope Amazon is prioritizing the fix.

>When Heroku went down, I sat around wishing it would come back up, and eventually it did.

This method of recovery just doesn't cut it in businesses with lots of transactions 24/7. Just sitting around can be flushing thousands per minute down the drain. In those deployments, you need to build across multiple cloud providers; and once you've reached that abstraction, you might as well have your own hardware as well and just burst to providers.

A company like Netflix can get by because they have their own hardware for streaming so an outage only affects people browsing for a movie at the time. And on top of that, their only interactive transaction is signing up for the service, which is extremely infrequent compared to the rest of the traffic. So essentially it's safe for Netflix to use AWS because they don't need high reliability/control of their own destiny.

>A small business with a single part-time developer can deploy an app to Heroku, and not need an ops person.

This use case I understand. However, anything at the scale that is frequently discussed in these contexts (gitlab, etc), they need ops people.

Re: Why we are not leaving the cloud

#144

Post by hueving »

dragonwriter wrote: post #103
tw04 wrote: post #98

Earlier quoted context omitted.

It isn't, and it's not even close. Unless you're running at less than 40% efficiency the cloud is more expensive for infrastructure running 24/7. If you're a shop with 1 server and no IT guy? Sure, probably makes sense to use the cloud - although I'd argue you'd be better off with managed hosting so you actually have a number to call if something breaks. An organization of even medium size? Not a chance.

> Unless you're running at less than 40% efficiency the cloud is more expensive for infrastructure running 24/7. Sure, paying extra for a dynamically scalable service is inefficient if your resource needs are flat 24/7/365, but is that realistically the case? (OTOH, one must not forget that there is a cost to engineer systems to take advantage of dynamic scaling to minimize costs while meeting requirements.)

>Sure, paying extra for a dynamically scalable service is inefficient if your resource needs are flat 24/7/365, but is that realistically the case?

Yes, the vast majority of apps are not 12 factor architectures designed for scaling so nodes can't be brought online and killed at the flip of a switch.

Re: Why we are not leaving the cloud

#145

Post by keepper »

morgante wrote: post #74
keepper wrote: post #68

Earlier quoted context omitted.

Netflix runs their most expensive part (pushing those terrabits of traffic ), with their own hardware and network. https://openconnect.netflix.com/en/ so, you were saying? Or do you think facebook,google,aol,tripadvisor and pretty much anyone beside HotNewStartup are incompetent?

I know Netflix runs their own edge nodes, which is precisely why OP sounds incompetent and uninformed. Netflix has spent lots of time and effort into analyzing and building the architecture which is optimized for them. > Or do you think facebook,google,aol,tripadvisor and pretty much anyone beside HotNewStartup are incompetent? Far from it, and that's precisely my point. At Facebook scale, it certainly makes sense to…

I bet to wager Netflix and Snapchat are at Facebook scale. Certainly $500M/ year in cloud commitments for the next 4 years seem to indicate so.

Re: Why we are not leaving the cloud

#146

Post by hueving »

morgante wrote: post #49
dsr_ wrote: post #40

Earlier quoted context omitted.

The cloud isn't a money-saving tactic. It enables you to test demand for a service for a very low absolute cost. Spin up a tiny machine for $60/year? And you can pay for it by the month, or the hour? That's great. You can find out if anybody is going to buy your thing at all. You can probably get to ramen-profitable. As soon as you can confidently predict sufficient demand, the economically rational decision is to hi…

> and the costs of cloud service are zooming up, potentially leaving you without the margin to invest. What do you mean by this, because it sounds like a straight up lie? Amazon and Google are in an ongoing price war and cut costs regularly.

>What do you mean by this, because it sounds like a straight up lie?

There is no economy of scale with the cloud. As you scale up your service, Amazon/Google happily scale your bill right up with it. When you roll your own datacenter, things get much cheaper per unit (bandwidth, storage, etc) as you get larger.

Re: Why we are not leaving the cloud

#147

Post by omgdresser »

cookiecaper wrote: post #126
bdamm wrote: post #44

Earlier quoted context omitted.

As a professional that came from software engineering let me tell you... that plugging in one machine is easy. Plugging in 150 machines is significantly more challenging. Providing disaster response, backups, stable power, and VLANs to secure services, quickly ratchets up the cost. And nobody wants to pay one engineer per machine to babysit one machine, x 150. That wouldn't even be a good idea, because engineers get…

First, most of the time it's not a question of paying someone to run an entire data center, but rather some racks in someone else's data center. The data center has data center staff who handle a lot of this. It's unfair to pretend that going bare metal entails starting with an empty plot of land and building/managing a whole data center. While that may be the case for the largest companies, even moderately-sized com…

I was about to make a comment similar to this, but this is articulated so much better than what I was able to get out.

I'm in agreement, after the hardware deploy, you're pretty much left with what you would have in a cloud environment: a bunch of VMs (or in this case OS instances) that need care and feeding, and only rarely, much rarely than many would think, do you have to replace a disk, or recable because requirements changed.

Re: Why we are not leaving the cloud

#148

Post by keepper »

mynameisvlad wrote: post #86
keepper wrote: post #68

Earlier quoted context omitted.

Netflix runs their most expensive part (pushing those terrabits of traffic ), with their own hardware and network. https://openconnect.netflix.com/en/ so, you were saying? Or do you think facebook,google,aol,tripadvisor and pretty much anyone beside HotNewStartup are incompetent?

More than likely so that the video feed is as close to you as possible, rather than only because of cost efficiency. The video is the most important part of their product, so keeping it at your ISP or a local exchange not only lowers latency, but most likely also gives better throughput since it only has to travel through your ISP's network or local exchange links without having to go through the rest of the internet…

Its not "more thank likely", it is the real reason. CDN's could not keep with their volume AS WELL as pricing structure.

Latency ( 30ms vs 300ms) doesn't matter much in video playback, only throughput. Once you hit play, it isn't bi-directional.

But you're just nitpicking. This IS their core business, and they realized its "too core" to give to someone else.

So yes, you do "outgrow the cloud", and well, part of "outgrowing" can be cost structure. Gitlab offers a free service, their main competitor, github, runs their own datacenter. Why do you think that is?

Re: Why we are not leaving the cloud

#149

Post by keepper »

scott_s wrote: post #48
dasil003 wrote: post #37

Earlier quoted context omitted.

> This may seem harsh, but relying on random commenters shows a huge flaw in how you guys went about this. Your other points and experience are well taken, but this is definitely too harsh. Think about the nature of GitLab as an open company building dev tools and paying below-market salaries. They have a huge amount of developer mindshare relative to their internal head count. They are not very competitive for hirin…

It's not just too harsh, it's not true. See the comments from those involved in the decision in this thread.

Why is it not true? Their main competitor, github, runs their own datacenter? Why is that?

Re: Why we are not leaving the cloud

#150

Post by qaq »

zip1234 wrote: post #106
deegles wrote: post #52

Earlier quoted context omitted.

> Hire a good ops team I feel this is one of those "easier said than done" statements...

Also can be very expensive. If another IT devops type costs 100k per year. How much will that buy in cloud services and time saved?

150 TB/month of data transfer on AWS will cost you 15k/ month the same thing on dedicated provider will be 500/month. And so on.
Post reply on HN