Live data from Hacker News

What job interviews taught me about Kubernetes

notnotp.com

151–160 of 233 posts

Re: What job interviews taught me about Kubernetes

#151

So I’m personally a huge fan of k8s and while I agree it may be „complicated”, it’s because deploying applications is complicated. (I want to point out that there is no requirement no set up cert manager, ArgoCD, external secrets, etc. - and many people who’d consider a VPS would happily slap a .env with an unencrypted secret then ssh to update, but when they choose Kubernetes they take the long route of doing proper…

> many people who’d consider a VPS would happily slap a .env with an unencrypted secret then ssh to update

I just want to point out that you can totally still do this with Kubernetes. Of course it's not correct, but you can save that unencrypted secret in a .env file right into your container while you're building it - no need to use Kubernetes's support for supplying environment variables from the manifest. And of course, you don't even need a Dockerfile to build that container - you can just exec into a running container, paste it in, and then docker save.

Kubernetes doesn't save you from making stupid decisions, it just makes it easier to make better ones.

Re: What job interviews taught me about Kubernetes

#152

> I still don't totally get why the shift happened when it did. Five years ago all three camps were doing fine. Now the VM+systemd crowd has basically disappeared from job postings, serverless stayed niche, and K8s just won. > > My best guesses: managed K8s (EKS, GKE, AKS) got mature and the talent pool flipped: enough people learned it that hiring for anything else became the harder choice. And Helm made "just use s…

What I've seen more than anything else is that Kubernetes built an ecosystem (of contributors and users, but also of companies invested in its success) that none of its competitors could or would. There was apparently a faction within Google that believed open-sourcing Kubernetes was a mistake because Google would have made more money keeping it in-house, but in terms of the success of the project I think it was entirely the right call, as was creating a foundation to maintain and promote it. Look at the history of its competition:

* DC/OS was always its own thing and as time went on, eventually Mesosphere was basically the sole maintainer of the underlying Mesos. Very little external contribution.

* OpenShift was different from Mesos and basically maintained only by Red Hat from the Makara acquisition (sometime in 2010 I think) to about mid-2015 (i.e. the point where they ripped out most of the OpenShift-native process isolation and orchestration and replaced it with Docker and Kubernetes). Pre-Kubernetes OpenShift frankly struggled to catch on and again, basically everybody who cared about developing it worked for one company.

* CoreOS was developing fleet in the open but dropped it outright when Kubernetes was released. The phrase I heard there was "We started to say something and Google finished our sentence." They pivoted to Kubernetes for orchestration so hard it was kind of awkward talking to customers who used fleet after that. In theory somebody could have picked it up like Kinvolk picked up rkt for awhile (and later CoreOS Linux as Flatcar), but as far as I know nobody ever made a serious effort to do so.

* Docker released Docker Swarm shortly after Kubernetes was released -- yet another one-company product. (I still don't really understand why they released Swarm -- for simple workloads, Docker Engine and Docker Compose were enough, and for more complex ones Docker Engine was, at that time, still the sole underlying runtime in Kubernetes. There were already two distinct orchestrators on the market, one from a much larger company with a lot more operational experience running containerized workloads than Docker had. What was their thought process?)

* HashiCorp released Nomad well after Kubernetes but not only was it another sole-corporate-maintainer orchestrator, it deliberately omitted a lot of the basics Kubernetes included like service discovery in an effort to stay simple -- so in very few cases was Nomad alone actually enough to orchestrate workloads (nor was it intended to be, as the Nomad engineers in the ~1.0 days would have been first to tell you). Past a point this made Nomad more work to get running and keep running than Kubernetes was.

The flip side is, I don't think a purely community-developed orchestrator would have won, even with a foundation backing it. It's not the corporate backing that's the issue, it's the lack of diversity in that corporate backing.

Re: What job interviews taught me about Kubernetes

#153
post #66

Here's my conspiracy theory -- There's a certain type of engineer (maybe 25% of them) who does "hype-driven-development." No matter the technology, they are huge advocates for the technology. The hype may be absolutely real, complete nonsense (e.g. mongodb), or somewhere in between (ai). The vast majority of the time it's hype for a new technology that feels 90% the same from the end-user perspective (react vs vue, d…

Tangential but IME those engineers do very very well, as they're good at marketing themselves, but more importantly they often align with the CTO/CEO who wants to present the company as the hottest thing since sliced bread – they signal they're on the same page.

Who is going to criticise the engineer who is open to new things / appears innovative / loves MVPs / 110% supportive of the latest bs the CTO is spewing? Yeah

Re: What job interviews taught me about Kubernetes

#154

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

Docker swarm is that simple solution you're looking for. But people don't need simple solutions. They want scalable solutions and Kubernetes fits this niche perfectly. You can deploy it on single server today and scale to 100 servers managed cluster tomorrow. Just to provide a similar example. Linux system is insanely complicated. Kernel alone has thousands of options. Distos have tens of thousands of packages. Where…

[deleted]

Re: What job interviews taught me about Kubernetes

#155

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

Is the simpler thing not Lambda/Cloud Run (with terraform)?

On the application developer side k8s is awesome fo, but the you look inside the box and it melts your face off.

I'm not sure a middle ground exists unfortunately. It's either full service like Lambda or bag of knives like k8s.

Re: What job interviews taught me about Kubernetes

#156

Earlier quoted context omitted.

From the article: > My personal threshold would be the moment the CTO isn't the only engineer anymore. As soon as a second person shows up, the problems K8s solves become real.

What a world we live in with people thinking that the problems of a 2 (founder) engineers startup deserve k8s complexity... Even with LLMs, they are going to rack up tech debt if their focus is - as it should in as mall startup - the final product and not the tech stack itself.

> Even with LLMs, they are going to rack up tech debt

And if they don’t?

Re: What job interviews taught me about Kubernetes

#157

Earlier quoted context omitted.

I thought that was the point of the article, right? That the tech benefits may not be there, but they’re using it for the non-tech benefits

> That the tech benefits may not be there, but they’re using it for the non-tech benefits My read of the article is that this is correct, but that the benefits they're using it for are the operational, and organisational. I think the comment you're replying to is arguing that those benefits don't really matter or outweigh the additional complexity costs when N=2 (engineers). I think I'd probably agree.

I think I still disagree.

If K8s is new to you, sure. Definitely not the time to learn it.

But I can see a world where it’s fine to use early on.

Especially if your team is cloud native. K8s isn’t really a new controversial toy in my eyes, it’s pretty well supported and good enough for most things out of the box.

I just don’t think it’s as big a deal as the “CTO IS WASTING EVERYONES TIME” argument.

Re: What job interviews taught me about Kubernetes

#158

Even as a solo dev there's generally been a yawning gap between k8s and manual infra that nothing has ever filled that well and it's part of why things like Heroku were so popular for a while.

From what I've seen, Azure container apps (ACA) sits exactly there in the middle: a managed Kubernetes, opinionated and full of defaults you may or may not like, but makes the deployment and maintenance a breeze.

Re: What job interviews taught me about Kubernetes

#159
post #10

One year ago I might agree that Kubernetes is an overkill but today? Ask your favorite GPT to generate manifests, get primary app into cluster with telepresence or execute straight from container and switch contexts and clusters like it's 90s again. One reason I dislike Docker Compose and Docker is lack of isolation. Yes sure if you put your arm deep enough you can get it, but on local k8s I can spin cluster per work…

Interesting. I have just started reading about Kubernetes. Is there an reading material that goes over this process you just described?

If you wanted to learn how to setup your own Kuberbetes cluster I found EasyK3S https://easyk3s.dev/ to be a good place to learn the basics.

It is not perfect, but a good place to start to get a hang of how to setup your own K8S setup if you are new to Kubernetes.

Re: What job interviews taught me about Kubernetes

#160
post #23
post #10

One year ago I might agree that Kubernetes is an overkill but today? Ask your favorite GPT to generate manifests, get primary app into cluster with telepresence or execute straight from container and switch contexts and clusters like it's 90s again. One reason I dislike Docker Compose and Docker is lack of isolation. Yes sure if you put your arm deep enough you can get it, but on local k8s I can spin cluster per work…

Strong agree, if there's one thing LLMs are excellent at, it's writing Terraform and Kubernetes deployments (and/or helm charts). What used to be half a day of research, trial and error, is now 20 seconds of AI churn and 98% of the time it nails it on the first try. And then point it at grafana and tell it to write you a dashboard for the new service/s. Easy peasy lemon squeezy. What used to require a team of 4 devop…

The other 2% your entire prod deployment gets deleted.
Post reply on HN