Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What job interviews taught me about Kubernetes

notnotp.com

Post reply on HN
Print view View original

71–80 of 233 posts

Re: What job interviews taught me about Kubernetes

#71

Post by jaggederest »

mikeocool wrote: post #63
zzyzxd wrote: post #49

Earlier quoted context omitted.

> One thing that I think people new to it don’t realize is that it’s not at all batteries included - to get a basic managed cluster setup, you’re still going to be installing a bunch of additional controllers (ingress, cert-manager, external dns to start). And if you can do this again, what's your solution to reverse proxy, certificate management, DNS...etc? I guess you can docker-compose some custom stack on a singl…

I don’t have an answer I’m in love with today, I basically just want less moving parts. As for EKS, having to monitor and manually scale the built in DNS service or else my queries are just going to stop resolving is not the type of thing I expect to have to manage on a managed service. I see they have finally released autoscaling for CoreDNS, though it took them 6 years.

Accidental complexity and essential complexity. There is no working system that achieves all the stated aims with fewer parts. [1]

[1] https://en.wikipedia.org/wiki/No_Silver_Bullet

Re: What job interviews taught me about Kubernetes

#72

Post by zug_zug »

mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

> I think there is a real hole in the market for a simple solution

Unless of course, all of the busywork that comes with kubernetes IS the value (to the engineer). Perhaps a bunch of engineers know at some level that locking the company into an overcomplicated cloud-within-a-cloud setup that has all sorts of weekly issues and requires constant work gives them a lot of job safety that they wouldn't get if they just used an AWS autoscaling group and you're done for the next 5 years.

Because simpler solutions DO exist (like a loadbalancer in front of an autoscale group, and not making a giant SOA for an app that orders you taxis, or books you a bnb or whatever nonsense).

Re: What job interviews taught me about Kubernetes

#73

Post by zbentley »

mschuster91 wrote: post #36
JohnMakin wrote: post #8

Earlier quoted context omitted.

If you struggle with any of that (a lot of what you listed is not strictly necessary to running managed kubernetes, specifically EKS) you are also going to struggle with a lot of other things on AWS, or wrangling a VM setup at any kind of scale.

> a lot of what you listed is not strictly necessary to running managed kubernetes, specifically EKS Oh it's not necessary per se but if you want to host a web service with any sort of state and not having to do stuff in parallel either by hand or by terraform, I'd consider the integrations pretty vital. It's easy enough (well, it's still addons whose versions you have to keep updated each on their own) once it is se…

I think the best supported and most mature pattern on most big cloud providers is precisely

> do stuff in parallel either by hand or by terraform

…specifically by terraform. Making k8s own the provisioning and management of external infrastructure on principle (as opposed to when that makes sense, e.g. load balancers/gateway/CSI providers) is not a good approach. Sure, it feels unified, but the cost of unification is incredibly not worth it.

Re: What job interviews taught me about Kubernetes

#74

Post by zug_zug »

zbentley wrote: post #69
zug_zug wrote: post #66

Here's my conspiracy theory -- There's a certain type of engineer (maybe 25% of them) who does "hype-driven-development." No matter the technology, they are huge advocates for the technology. The hype may be absolutely real, complete nonsense (e.g. mongodb), or somewhere in between (ai). The vast majority of the time it's hype for a new technology that feels 90% the same from the end-user perspective (react vs vue, d…

The linked article discusses very different reasons for preferring kube. CTOs and hiring managers like it for reasons totally different from the cargo cult/hype-driven engineers.

Yeah I read the article and I saw that.

And I do think there is a way to use kubernetes with minimal damage, but it requires making firm rules about not focusing on things that aren't needed yet (e.g. istio) and making firm hiring choices about only people who understand that such optimizations are complete wastes of time for a series A startup.

Re: What job interviews taught me about Kubernetes

#75

Post by mikeocool »

josevalerio wrote: post #65
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

Dear friend, you have built a Kubernetes https://www.macchaffee.com/blog/2024/you-have-built-a-kubern...

There was once a time when we could deploy software without spinning up 3 etcd databases, multiple controller processes constantly running event loops, and a virtual networking layer, before you even get off the ground.

Perhaps those days are behind us.

Re: What job interviews taught me about Kubernetes

#76

Post by ajayvk »

BobbyTables2 wrote: post #29
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

Pretty sure if there was a simple alternative, people would hate it. Everyone initially wants thing A. But then they want to customize it to do all permutations and combinations n of A, B, C. They want it to be extensible. They want redundancy. They want orchestration. They want integration. It’s why practically every config file format eventually becomes its own scripting language. Even HTML started off simple — now…

I don't think you can provide all the features of Kubernetes while reducing the complexity. What is possible is to support a subset of the features of Kubernetes while making it easy to use.

https://github.com/openrundev/openrun is a project I am building. It supports declarative deployments, on a single-node with Docker or onto Kubernetes. The target use cases is limited to standalone web app, like internal tools. No support for stateful services, you manage stateful services yourself. With that simplification, OpenRun provides a much easier developer experience.

Re: What job interviews taught me about Kubernetes

#77

Post by foo4u »

zzyzxd wrote: post #49
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

> One thing that I think people new to it don’t realize is that it’s not at all batteries included - to get a basic managed cluster setup, you’re still going to be installing a bunch of additional controllers (ingress, cert-manager, external dns to start). And if you can do this again, what's your solution to reverse proxy, certificate management, DNS...etc? I guess you can docker-compose some custom stack on a singl…

I've been building multi-cluster Kubernetes for some time and things like External DNS and Ingress controllers per app are just non-starters. They always felt kludgy having K8S orchestrate things external to the cluster and their anti-patterns IMO.

Re: What job interviews taught me about Kubernetes

#78

Post by SOLAR_FIELDS »

ajayvk wrote: post #76
BobbyTables2 wrote: post #29

Earlier quoted context omitted.

Pretty sure if there was a simple alternative, people would hate it. Everyone initially wants thing A. But then they want to customize it to do all permutations and combinations n of A, B, C. They want it to be extensible. They want redundancy. They want orchestration. They want integration. It’s why practically every config file format eventually becomes its own scripting language. Even HTML started off simple — now…

I don't think you can provide all the features of Kubernetes while reducing the complexity. What is possible is to support a subset of the features of Kubernetes while making it easy to use. https://github.com/openrundev/openrun is a project I am building. It supports declarative deployments, on a single-node with Docker or onto Kubernetes. The target use cases is limited to standalone web app, like internal tools. N…

I look forward to the evolution of your project into a less standardized Kubernetes as end users request more and more features of your project.

Re: What job interviews taught me about Kubernetes

#79

Post by mickael-kerjean »

Thaxll wrote: post #55
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

k8s is not a pain, I would never return to something like Pupet / Ansible / etc ... to deploy bare ec2 instances, it's just re-inventing the wheel badly. Just use ECS / Fargate with an ALB in front if you need a simpler use case.

I've had the opposite experience. I used to run k8s on bare metal, troubleshooting something at least once a month (DNS going down was a recurring favorite). The breaking point came with the churn in the ecosystem, got bitten by the deprecation of the community darling weave net cni plugin, the killing of the nginx ingress was the nail in the coffin knowing I had way too many annotation tight to the ingress that it would take longer to migrate those than go the ansible way with k8s imposing tight upgrade schedules. While I agree ansible feels a lot more dirty than k8s, I spend much less time on infrastructure, sleeping betterat night and handling things like databases is much simpler

Re: What job interviews taught me about Kubernetes

#80

Post by Eridrus »

mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

We started out core product on ECS, which is a declarative way to run a containerized service. It has been nice and reliable, but it has limitations (slow scaling, weird AWS Quotas if you have ephemeral tasks).

We're moving our non-critical components onto EKS (pipelines, tooling, etc). We had one outage from runaway IP allocation in a subnet, but otherwise it's been pretty stable.

I do hear vague horror stories so I'm really not excited about moving our prod stack to it, but it's actually been really good for installing 3rd party software so far.

Post reply on HN