Rolling your own servers with Kubernetes • Hacker News Board

Re: Rolling your own servers with Kubernetes

#61

Post by acdha » Mon, May 06, 2019, 5:25 PM UTC

gambler wrote: post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

> The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to afford their own infrastructure.

That conspiracy theory depends on not asking about the major costs areas you left out (e.g. ops, security, reliability) which are the most significant until you're at a fairly large scale. The places I saw switching to cloud providers did so after comparing the quality & cost of the options available, not because they were following the cool kids.

Re: Rolling your own servers with Kubernetes

#62

Post by [deleted] » Mon, May 06, 2019, 5:26 PM UTC

segmondy wrote: post #59

I use to enjoy having a cheap desktop under my table or in a closet serving traffic to people across the internet. Computers have gotten faster, software has gotten better, network has gotten faster, things have gotten cheaper. Sadly instead of seeing more of these, most of these are now being outsourced to cloud providers, we have bought and drank the kool-aid that they can do it better and cheap. Which is not true,…

[deleted]

Re: Rolling your own servers with Kubernetes

#63

Post by fooqux » Mon, May 06, 2019, 5:27 PM UTC

isatty wrote: post #43

falcolas wrote: post #32

Earlier quoted context omitted.

> spend weeks arguing with AWS support Yup. We've run into that repeatedly. The "we didn't notice anything on our side, please send more screenshots and logs" gets really old when working with "managed services". Network packet losses/truncations, EKS control plane failures, cloudformation stacks getting stuck in really wierd states, inconsistent cloudformation implementations for new and existing services, ENI weird…

Cloudformation is a tragedy in itself.

It's infinitely better than what Azure provides, IMO.

Re: Rolling your own servers with Kubernetes

#64

Post by kevin_nisbet » Mon, May 06, 2019, 5:29 PM UTC

the-rc wrote: post #36

q3k wrote: post #12

Earlier quoted context omitted.

I highly recommend Rook [1], which is based on Ceph, provides PersistentVolumes to k8s workloads and is also running on k8s itself. [1] - https://rook.io/

I worked on Colossus at Google and Ceph is the closest thing out there. Gregory Farnum gave a great talk about it at the Open Source Summit 2017. I heard from someone at a large company, though, that it's not getting a lot of love from Red Hat nowadays, even if you're a large paying customer. Now I'm curious.

I'm curious also. I don't have any special insight into RedHat, but rook went 1.0 just last week: https://blog.rook.io/rook-v1-0-a-major-milestone-689ca4c7550...

That announcement gave me the impression the project is making active progress.

Re: Rolling your own servers with Kubernetes

#65

Post by kortilla » Mon, May 06, 2019, 5:36 PM UTC

acdha wrote: post #61

gambler wrote: post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

> The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to afford their own infrastructure. That conspiracy theory depends on not asking about the major costs areas you left out (e.g. ops, security, reliability) which are the most significant until you're at a fairly large scale.…

Ops, security, and reliability don’t go away when using EC2 either, which is what led the massive cloud explosion.

Places switched because they didn’t like having to forecast capacity so hourly billed resources turned on and off at the drop of a hat were super attractive.

Re: Rolling your own servers with Kubernetes

#66

Post by spamizbad » Mon, May 06, 2019, 5:36 PM UTC

gambler wrote: post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

There's no conspiracy: operating your own hardware in a datacenter requires expertise, start-up costs, and additional time. That power, control and (later) cost-savings doesn't come without a few drawbacks.

Short-term budgets and time windows to meet deliverables shrank along with the costs. That's why so many are on the cloud.

Re: Rolling your own servers with Kubernetes

#67

Post by duxup » Mon, May 06, 2019, 5:39 PM UTC

geezerjay wrote: post #51

SteveNuts wrote: post #38

Earlier quoted context omitted.

Meanwhile the market and mindshare has moved on to the next shiny new tech.

Kubernetes has been here for a while. I'm sure we can agree that it ceased to be a candidate for a meteoric fad for a while now.

I am not a Kubernetes expert but I feel like Kubernetes still has such a high rate of change going on that ... it still is in that phase where you just don't know what you might get out of it... maybe not "just a fad" but still variable enough that who knows where we'll be or if in two years it's all different.

Re: Rolling your own servers with Kubernetes

#68

Post by duxup » Mon, May 06, 2019, 5:42 PM UTC

gambler wrote: post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

I worked in a shop that had dedicated lab equipment and resources.

Suddenly buying new equipment that we would use for years became like pulling teeth, no money they said. They couldn't explain what changed.

Then just as suddenly they were willing to approve spending more than our one time equipment purchase... per month in monthly cloud costs / some poorly thought out cloud deployments....

Result:

-1 Dedicated person to manage the lab.

+2 two new way more expensive dev ops guys.

-X Occasional new equipment costs / service contracts.

+Y A metric ton of reoccurring monthly costs.

Re: Rolling your own servers with Kubernetes

#69

Post by rhinoceraptor » Mon, May 06, 2019, 5:43 PM UTC

gambler wrote: post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

The hardware isn’t the expensive part, it’s the people to run it that is.

Re: Rolling your own servers with Kubernetes

#70

Post by john_moscow » Mon, May 06, 2019, 5:46 PM UTC

lima wrote: post #11

argd678 wrote: post #7

Last physical datacenter my team at the time ran our app in we spent weeks troubleshooting some hardware network driver issues that caused the network to drop. Was an enormous distraction and Dell and VMware support were useless at resolving it for us. I’m glad to have others deal with those low level “oh it must just be your setup” issues.

Anecdotal evidence. I've also seen colleagues spend weeks arguing with AWS support while debugging a weird performance degradation issue, that would have been straight-forward to investigate in a bare metal deployment with full control over everything. It's not like the cloud is a magical place where no unexpected issues ever happen. Cloud providers can be surprisingly buggy, especially AWS, particularly at scale whe…

You need to treat a commodity like a commodity. It's perfectly OK to run business-critical processes in someone's cloud, as long as you have a fully automated process that will deploy your entire environment from the latest backup to any set of cloud nodes at any cloud provider you point it to. Unless you're doing that, you're just shifting risks, not eliminating them.