Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What job interviews taught me about Kubernetes

notnotp.com

Post reply on HN
Print view View original

41–50 of 233 posts

Re: What job interviews taught me about Kubernetes

#41

Post by epgui »

tayo42 wrote: post #30
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

Isn't fargate or ecs that simple service?

I find them just as complicated as k8s.

Re: What job interviews taught me about Kubernetes

#42

Post by suralind »

So I’m personally a huge fan of k8s and while I agree it may be „complicated”, it’s because deploying applications is complicated. (I want to point out that there is no requirement no set up cert manager, ArgoCD, external secrets, etc. - and many people who’d consider a VPS would happily slap a .env with an unencrypted secret then ssh to update, but when they choose Kubernetes they take the long route of doing proper GitOps and complain that there are so many things to configure :)

But I found funny that the OP summarized to use Kubernetes when CTO is no longer the only dev.

Re: What job interviews taught me about Kubernetes

#43

Post by himata4113 »

mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

I don't know... running a startup sized kubernetes is relatively easy and pain free these days (k3s). Especially when it comes to scaling up.

CNPG is an absolute monster (in a good way). cert-manager is easier than the docker alternative, calico has never failed me (except in bgp mode which has some footguns like not being able to come back from a dead state since it has a chicken and an egg problem unless you point it to the external load balancer which I would have known if I read the documentation). trafeik is all you need. talosos largely mitigates the bare metal problems and comes pre-hardened and pre-optimized.

I solo most of my development projects and have used k3s for all of them. The only complaint is that cert-manager by default will fail silently and your certificates will expire. I largely mitigated this by having proper visibility setup via grafana and automated alerts (warns if certificates are about to expire) which should have been done by me anyway.

Two years ago I'd agree, today with LLMs everything I have runs talos with fully automated updates and I haven't had to be on-call for almost a year.

Re: What job interviews taught me about Kubernetes

#44

Post by reillyse »

The reason this is accelerating recently is agents are really good at spinning up k8s clusters. They've made devops work super super simple. Basically all the annoying stuff you know you should do but it's way too much hassle - using let's encrypt to create unique certs for every app in your cluster to enable zero trust, configuring permissions and security profiles for everything etc etc (never mind just standing them up in the first place) - it's all simple now.

Re: What job interviews taught me about Kubernetes

#45

Post by nyrikki »

embedding-shape wrote: post #18
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

> I think there is a real hole in the market for a simple solution that lets you deploy some containers to some instances in a declarative fashion without all of that complexity and does decent LTS versions Hashicorp's Nomad basically is just that, supports various way of running stuff too which is neat. Shame about the license change which basically killed all my interest in it, so seems the hole is indeed still unf…

For simple cases I just launch podman containers on long lived hosts with ansible.

You can still add pods if needed and the systemd integration works.

Plus you can actually improve isolation by co-hosting services under separate UIDs.

Like any container it is just co-hosting, and elasticity is a bit slower with autoscaling instances, but it removes most of the complexity of K8s which very few org benefit from or have the culture to support.

Re: What job interviews taught me about Kubernetes

#47

Post by phailhaus »

reillyse wrote: post #44

The reason this is accelerating recently is agents are really good at spinning up k8s clusters. They've made devops work super super simple. Basically all the annoying stuff you know you should do but it's way too much hassle - using let's encrypt to create unique certs for every app in your cluster to enable zero trust, configuring permissions and security profiles for everything etc etc (never mind just standing th…

The setup may be simple, but what about the maintenance?

Re: What job interviews taught me about Kubernetes

#48

Post by mocamoca »

embedding-shape wrote: post #18
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

> I think there is a real hole in the market for a simple solution that lets you deploy some containers to some instances in a declarative fashion without all of that complexity and does decent LTS versions Hashicorp's Nomad basically is just that, supports various way of running stuff too which is neat. Shame about the license change which basically killed all my interest in it, so seems the hole is indeed still unf…

As CTO of a small startup and cutting costs, setting up hashicorp nomad + bare metal is a joy to work with.

Some self-reloading HAProxy in nomad to automatically assign URLs to services when needed. Could have used Consul but meh.

Tailscale for private networking.

Re: What job interviews taught me about Kubernetes

#49

Post by zzyzxd »

mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

> One thing that I think people new to it don’t realize is that it’s not at all batteries included - to get a basic managed cluster setup, you’re still going to be installing a bunch of additional controllers (ingress, cert-manager, external dns to start).

And if you can do this again, what's your solution to reverse proxy, certificate management, DNS...etc? I guess you can docker-compose some custom stack on a single machine, maybe add one more machine then you can say it's HA enough for small scale. But you can also spend the same amount of time to install those kubernetes controllers with zero customization. In my experience, if you go with the default configuration, most of the well-maintained k8s components are boring as hell these days.

> (if you’re on EKS, make sure to read about scaling and monitoring CoreDNS)

If load to your service increases, you need to scale up/out your service. This is universally true. Do you have a proprietary solution that's easier and more reliable than bumping up the replicas count in kubernetes?

There are lots of design decisions in Kubernetes that I hate. But if you want me to choose between Kubernetes and any proprietary stack, in 2026, I would definitely choose Kubernetes.

Re: What job interviews taught me about Kubernetes

#50

Post by mocamoca »

mikeocool wrote: post #24
embedding-shape wrote: post #18

Earlier quoted context omitted.

> I think there is a real hole in the market for a simple solution that lets you deploy some containers to some instances in a declarative fashion without all of that complexity and does decent LTS versions Hashicorp's Nomad basically is just that, supports various way of running stuff too which is neat. Shame about the license change which basically killed all my interest in it, so seems the hole is indeed still unf…

Yeah I’ve always meant to check out nomad and never had an opportunity. Though as I recall, it makes heavy use of consul, which I have used in anger, and makes me a little weary (though that experience is likely very out of date).

I've been using Nomad for years without Consul. Maybe if you complex networking requirements it is worth it, else don't really need it.
Post reply on HN