Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

news.ycombinator.com

Post reply on HN
Print view View original

341–350 of 433 posts

Re: Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

#341

Post by mynameisash »

Reading the comments here validates my experience. When K8s was pitched as a way to make this all run smoothly, I thought, "Great! I'll write my code, specify what gets deployed and how many times, and it'll Just Work(tm)." I built a service which had one driver node and three workers. Nothing big. It deployed Dask to parallelize some compute. The workload was typically ~30 seconds of burst compute with some pretty minor data transfer between pods. Really straightforward, IMO.

Holy smokes, did that thing blow up. A pod would go down, get stuck in some weird state (I don't recall what anymore), and K8s would spin a new one up. Okay, so it was running, but with ever-increasing zombie pods. Whatever. Then one pod would get in such a bad state that I had to nuke all pods. Fortunately, K8s was always able to re-create them once I deleted them. But I was literally deleting all my pods maybe six or seven times per day in order to keep the service up.

Ultimately, I rewrote the whole thing with a simplified architecture, and I vowed to keep clear of K8s for as long as possible. What a mess.

Re: Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

#343

Post by dublin »

hpkuarg wrote: post #90
jmillikin wrote: post #33

Earlier quoted context omitted.

Kubernetes can't (currently) scale to Google sizes. It's designed for small- or medium-sized businesses, which might have 50,000 VMs or fewer. There are entire SaaS industries that could fit into a single Google/Facebook/Amazon datacenter.

> There are entire SaaS industries that could fit into a single Google/Facebook/Amazon datacenter. Forget a whole datacenter, even just one rack is an unimaginable amount of computing power, these days!

Fact: An well-equipped Raspberry Pi 4 has more memory, more compute power, more storage and vastly faster networking than the Cray supercomputer I worked with at a major oil company in the 1990's! With the exception of the craziness around click-tracking (web ads and marketing have warped compute use even more than crypto), the data required to run even enterprise-scale businesses today is not really all that large.

For almost all purposes, we don't really need thousands of containers running on unimaginably fast computers, coordinated by AI-driven automation systems. What we need is software that is not morbidly obese.

Re: Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

#344

Post by dublin »

dusanh wrote: post #38
binarymax wrote: post #29

Kubernetes is a Google scale solution. Lots of teams said “hey if Google does it then it must be good!”…but forgot that they didn’t have the scale. It caught on so much that for whatever reason it’s now the horrendous default. I’ve worked on at least 3 consulting projects that incorporated K8s and it slowed everything down and took way too much time, and we got nothing in return - because those projects only needed s…

Asking as someone who has only tipped his toes into devops lately and is looking to learn K8, what is considered a reasonable "lightweight" alternative to Kubernetes these days?

Not an alternative, but if you want to get a feel for Kubernetes and managing various kinds of servers, check out KubeSail and their options for K8s/K3s

It's ridiculous overkill, but I'm looking at a NextCloud server on one of their PiBox hardware servers for the house. (You don't need a PiBox - their stuff will run fine on little instances from AWS/DigitalOcean/Hertzner, etc., or a spare PC you have lying around...)

Re: Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

#345

Post by adra »

mynameisash wrote: post #341

Reading the comments here validates my experience. When K8s was pitched as a way to make this all run smoothly, I thought, "Great! I'll write my code, specify what gets deployed and how many times, and it'll Just Work(tm)." I built a service which had one driver node and three workers. Nothing big. It deployed Dask to parallelize some compute. The workload was typically ~30 seconds of burst compute with some pretty m…

This can probably be chalked up to youre-doing-it-wrong (sorry) but not knowing your precise scenario, it's hard to know what went wrong. Maybe really old versions misbehaved (only started a few years ago and its been smooth sailing), but I've never seen your problem on any of our stuff and we have dozens of different services on a bunch of languages/frameworks, and none of them just give up for no reason ( though a lot often die for predictable and self-induced reasons).

I think there was some jank on AWS CNI drivers at one point that delayed pod init, but that's probably the most wtf that I've personally bumped into thankfully.

Re: Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

#346

Post by bostonsre »

jmillikin wrote: post #112
throwaway787544 wrote: post #93

Earlier quoted context omitted.

> but instead that operational load of running in-house software should be borne primarily by the developers of that software Go back and read a few DevOps books and blogs by the founders of it. We will always need separate disciplines for dev and ops, just like we need mechanical engineers and mechanics/racecar drivers. But we need them to work together and communicate better to solve problems better and not throw d…

My second job in large-scale software was at Google, which used the "DevOps model" since before DevOps was named. I have no need to read a blog on it. You want the person who designs the car to know what a car is , and to be able to diagnose basic issues like "the fuel gauge says 'empty' and engine won't start". And there's no analogy to an Indy car driver in software, every distributed system is self-driving. The mo…

> And there's no analogy to an Indy car driver in software, every distributed system is self-driving.

You typically build systems to sell a product. The driver is the customer that uses the app. You build the app to be as simple as possible to effectively operate while providing the best possible outcome for that customer. The driver doesn't care or need to know how the engine or the rest of the car works, just that the engine and car works well.

Re: Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

#348

Post by gh02t »

dublin wrote: post #343
hpkuarg wrote: post #90

Earlier quoted context omitted.

> There are entire SaaS industries that could fit into a single Google/Facebook/Amazon datacenter. Forget a whole datacenter, even just one rack is an unimaginable amount of computing power, these days!

Fact: An well-equipped Raspberry Pi 4 has more memory, more compute power, more storage and vastly faster networking than the Cray supercomputer I worked with at a major oil company in the 1990's! With the exception of the craziness around click-tracking (web ads and marketing have warped compute use even more than crypto), the data required to run even enterprise-scale businesses today is not really all that large.…

I mean a single V100 GPU (~= 100 TFLOPS) has FLOPS throughputs similar to top end mid-00's supercomputers like Blue Gene, at least superficially. And you can squeeze like 4+ in a 1U if you have enough cooling and power.

https://en.wikipedia.org/wiki/History_of_supercomputing

Re: Ask HN: If Kubernetes is the solution, why are there so many DevOps jobs?

#350

Post by russellendicott »

KaiserPro wrote: post #311
binarymax wrote: post #29

Kubernetes is a Google scale solution. Lots of teams said “hey if Google does it then it must be good!”…but forgot that they didn’t have the scale. It caught on so much that for whatever reason it’s now the horrendous default. I’ve worked on at least 3 consulting projects that incorporated K8s and it slowed everything down and took way too much time, and we got nothing in return - because those projects only needed s…

> Kubernetes is a Google scale solution The problem is that its _not_ a google scale solution. Its something that _looks_ like a google scale solution, but is like a movie set compared to the real thing. for example: https://kubernetes.io/docs/setup/best-practices/cluster-larg... no more than 5k nodes. Its extra ordinarily chatty at that scale, which means it;ll cost on inter-vpc traffic. I also strongly suspect that…

Yeah, I don't know if it's because ECS was my first container orchestration experience but every time I look at teams trying to do k8s on AWS I think how much easier ECS would be.
Post reply on HN