Live data from Hacker News

Rolling your own servers with Kubernetes

gravitational.com

11–20 of 209 posts

Re: Rolling your own servers with Kubernetes

#11
post #7

Last physical datacenter my team at the time ran our app in we spent weeks troubleshooting some hardware network driver issues that caused the network to drop. Was an enormous distraction and Dell and VMware support were useless at resolving it for us. I’m glad to have others deal with those low level “oh it must just be your setup” issues.

Anecdotal evidence. I've also seen colleagues spend weeks arguing with AWS support while debugging a weird performance degradation issue, that would have been straight-forward to investigate in a bare metal deployment with full control over everything.

It's not like the cloud is a magical place where no unexpected issues ever happen. Cloud providers can be surprisingly buggy, especially AWS, particularly at scale when you start hitting the implicit scaling limits their docs conveniently forgot to mention.

Each technology has their unique challenges.

Re: Rolling your own servers with Kubernetes

#14
post #7

Last physical datacenter my team at the time ran our app in we spent weeks troubleshooting some hardware network driver issues that caused the network to drop. Was an enormous distraction and Dell and VMware support were useless at resolving it for us. I’m glad to have others deal with those low level “oh it must just be your setup” issues.

In my last team, our team suffered from a CPU bug and a SSD bug that caused painful debugging nights. We used multithreading heavily, and it could have been library/kernel issues or some bugs in our code, which cover massive amount of potential target code that needed to sort through initially. I don't think small companies can afford such gigantic debugging effort, so either they should rely on their luck (or flaky tests) or on someone else who can afford it (a cloud/infra service provider or a startup that depends on you.)

Re: Rolling your own servers with Kubernetes

#16
isn't this what they all sell nowadays? A prettier UI on top of kubernetes?

- Kubermatic by loodse

- rancheros

- even mesosphere is kinda that now, except the first two are opensource, but definitely not the only solutions

I personally deploy kubespray here and there, but I would still recommend smartos for normal humans, unless they want to setup an elk + kafka cluster or whatever else you might want to do.

Re: Rolling your own servers with Kubernetes

#17
post #11
post #7

Last physical datacenter my team at the time ran our app in we spent weeks troubleshooting some hardware network driver issues that caused the network to drop. Was an enormous distraction and Dell and VMware support were useless at resolving it for us. I’m glad to have others deal with those low level “oh it must just be your setup” issues.

Anecdotal evidence. I've also seen colleagues spend weeks arguing with AWS support while debugging a weird performance degradation issue, that would have been straight-forward to investigate in a bare metal deployment with full control over everything. It's not like the cloud is a magical place where no unexpected issues ever happen. Cloud providers can be surprisingly buggy, especially AWS, particularly at scale whe…

I’ve had performance issues with GCP, however I can switch cloud providers in that case without too much effort. With physical hardware that’s much harder.

Re: Rolling your own servers with Kubernetes

#18
post #7

Last physical datacenter my team at the time ran our app in we spent weeks troubleshooting some hardware network driver issues that caused the network to drop. Was an enormous distraction and Dell and VMware support were useless at resolving it for us. I’m glad to have others deal with those low level “oh it must just be your setup” issues.

In my last team, our team suffered from a CPU bug and a SSD bug that caused painful debugging nights. We used multithreading heavily, and it could have been library/kernel issues or some bugs in our code, which cover massive amount of potential target code that needed to sort through initially. I don't think small companies can afford such gigantic debugging effort, so either they should rely on their luck (or flaky…

How is that related to bare metal vs. clouds? That could have happened on either.

Re: Rolling your own servers with Kubernetes

#19
post #15
post #13

The article seems to focus on K8s with reference to micro-services. How well does K8s do if you're running a monolith?

Using kubernetes without microservices is like using hadoop on a 1000 row csv file

Ah, as someone not well versed with K8s, thanks for that analogy.
Post reply on HN