Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What job interviews taught me about Kubernetes

notnotp.com

Post reply on HN
Print view View original

121–130 of 233 posts

Re: What job interviews taught me about Kubernetes

#121

Post by httgp »

mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

> there is a real hole in the market for a simple solution that lets you deploy some containers to some instances in a declarative fashion without all of that complexity and does decent LTS versions

There's Nomad for this; I wish more teams would run Nomad.

Re: What job interviews taught me about Kubernetes

#122

Post by [deleted] »

singron wrote: post #99
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

I really wish there was an 80% kubernetes. I think you could get there with some changes: 1. No overlay networks. 1 IP per machine. pods use dynamically allocated ports, and the kubelet enforces pods listen only on their assigned ports using seccomp. 2. No kube-proxy or equivalent Layer-4 "load-balancer". It's not good, but it's often used. You should use some kind of Layer-7 load balancing instead. Also you need to…

[deleted]

Re: What job interviews taught me about Kubernetes

#123

Post by stock_toaster »

ddreier wrote: post #104
singron wrote: post #99

Earlier quoted context omitted.

I really wish there was an 80% kubernetes. I think you could get there with some changes: 1. No overlay networks. 1 IP per machine. pods use dynamically allocated ports, and the kubelet enforces pods listen only on their assigned ports using seccomp. 2. No kube-proxy or equivalent Layer-4 "load-balancer". It's not good, but it's often used. You should use some kind of Layer-7 load balancing instead. Also you need to…

Unless you hate HCL, 1, 2, and 3 pretty much describe Nomad exactly. We run over 100k production applications on Nomad. But migration to AWS from private data centers, our HashiCorp bill, and the severe lack of Nomad talent, have finally pushed us to k8s (EKS).

I wish there was an EKS-like for Nomad!

Re: What job interviews taught me about Kubernetes

#124

Post by dsincl12 »

mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

Dokku?

Re: What job interviews taught me about Kubernetes

#125

Post by InvertedRhodium »

simoncion wrote: post #58
sshine wrote: post #27

Earlier quoted context omitted.

> less about K8’s and more about the infra as code movement. It doesn’t matter if you use K8, CDK, or terraform - you get the same benefits the OP stated I’d like to gently push back on that. ;-D Terraform, when committed to git, provides organisational memory. But less so uniformity, since all providers are different (and you should expect different things when applying). No tracing besides git. And tfstate is hard…

> And tfstate is hard to share between developers... Really? For years and years we put our tfstate files into private S3 buckets at $DAYJOB and it seemed to work just fine. We didn't even take pains to ensure that everyone was on the very same version of the Terraform CLI. What problems did you guys run into?

One person upgrading terraform can break the state format for backwards compatibility. We use direnv with asdf to ensure everyone is in the same version now.

Re: What job interviews taught me about Kubernetes

#126

Post by bandrami »

BobbyTables2 wrote: post #29
mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

Pretty sure if there was a simple alternative, people would hate it. Everyone initially wants thing A. But then they want to customize it to do all permutations and combinations n of A, B, C. They want it to be extensible. They want redundancy. They want orchestration. They want integration. It’s why practically every config file format eventually becomes its own scripting language. Even HTML started off simple — now…

It was supposed to primarily be a link target as well as possibly a source. Berners-Lee guessed the ratio of in to out wrong there.

Re: What job interviews taught me about Kubernetes

#127

Post by JohnMakin »

hadlock wrote: post #33
JohnMakin wrote: post #4

> I still don't totally get why the shift happened when it did. Five years ago all three camps were doing fine. Now the VM+systemd crowd has basically disappeared from job postings, serverless stayed niche, and K8s just won. > > My best guesses: managed K8s (EKS, GKE, AKS) got mature and the talent pool flipped: enough people learned it that hiring for anything else became the harder choice. And Helm made "just use s…

The upgrade cycle is a feature, not a bug. If (when) you need to do a big lift and shift, or there's some 0 day CVE, push buttan, get security update. You CAN drift behind but there's a real $$$ cost to that now. Every three months I toss opus at my k8s stack and verify it's compliant with k8s v1.xx.y and then push the upgrade button on my staging cluster, and then a week later I push the upgrade button on my prod cl…

The gateway api people have clearly won and I can’t truly complain because I’m not a maintainer, but I have contributed in the past to a nontrivial part of the tooling built off this ecosystem. The issues with snippets/annotations are a core deficiency with k8s design and eliminating this api creates more problems than it supposedly solves. I have been working on solutions of my own preparing for this inevitability, but it’s rough. ingress annotations like it or not run the modern infra tech stack. if they are persona non grata at any point in the future, a lot of people are going to have a lot of urgent consulting problems in the near to mid distant future.

I to this date have not seen a viable drop in replacement to how I’ve seen big orgs use the ingress controller stack with the gateway api and what i understand currently is ingate is basically DOA.

Re: What job interviews taught me about Kubernetes

#128

Post by secondcoming »

kilobaud wrote: post #97
busterarm wrote: post #38

Earlier quoted context omitted.

Nomad, Consul and Vault all running on VMs that you manage with Terraform. The problem is that when you run this long enough you want K8s features anyway.

And your starter “production” deployment of the Nomad/Consul/Vault stack is literally 12 VMs, comprising three independent Raft clusters. There is no decent way to do zero-downtime instance replacement without building your own orchestration layer, but also they’ve had a years-long track record of shipping bad upgrades and following up with only manual remediations or workarounds instead of a fix. As someone who has…

This is a timely post. We are going to use Consul to replace the need for Internal Load Balancers. What issues do you have with it?

Re: What job interviews taught me about Kubernetes

#129

Post by erpellan »

mikeocool wrote: post #14

I made this decision at a startup (albeit when the eng team was ~30 people, and we had a monolith with ~10 supporting services). I wouldn’t do it again, even for the reasons stated in the article. The uniformity is nice, we were moving from apps running directly ec2 instances provisioned with ansible. Each time we spun up a new service it was a process to get the ec2 instances provisioned just so. But k8s is such a p…

There was: Heroku

It was glorious.

Post reply on HN