Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A skeptic's first contact with Kubernetes

blog.davidv.dev

Post reply on HN
Print view View original

11–20 of 105 posts

Re: A skeptic's first contact with Kubernetes

#11

Post by deivid »

politelemon wrote: post #9

This was a useful read and somewhat gels with my experiences. Looking at the statement at the beginning > and it only requires you to package your workload as a Docker image, which seems like a reasonable price to pay. is no longer true as you continue down the path, since it actually requires you to do a lot more than you'd think.

I've not yet gone down that path, in which other ways does your workload need to adapt? I understand that you may want to get more value out of logs, and include side-cars, you may want inter-pod encryption and include service meshes, etc; but that'd be something that requires extra complexity in any setup

Re: A skeptic's first contact with Kubernetes

#12

Post by mtndew4brkfst »

jauntywundrkind wrote: post #3

> Why are the storage and networking implementations "out of tree" (CNI / CSI)? Given the above question, why is there explicit support for Cloud providers? eg: LoadBalancer supports AWS/GCP/Azure/.. Kubernetes has been pruning out vendor-specific code for a while now, moving it out of tree. The upcoming 1.31 release will drop a lot of existing, already deprecated support for AWS & others from Kubernetes proper. http…

Kustomize is builtin to kubectl & is being removed

Hadn't heard this until now, I'm a rather happy user after being jaded about Helm 2/3. Do you happen to know if this removal is sentimentally closer to "we don't want to keep maintaining kustomize" or to "we don't want to keep embedding kustomize, please use its binary directly"?

Re: A skeptic's first contact with Kubernetes

#13

Post by galdosdi »

hbogert wrote: post #7

His take on text interpolation is very right. I'm a SWE turned SRE because as a developer I really enjoyed using K8s. But as a full-time SRE where I work just means YAML juggling. It's mind numbing that everybody is okay with this, this really is our domain's assembly era, albeit with whitespace, colons, dashes and brackets. I've found solace in CUE which I just run locally to catch all the small errors everybody mak…

> Putting the CUE validation in our pipeline is too confronting for others

Sad. Can you get away with boiling the frog by getting it initially added but configured to check very, very little? Maybe some specific rare category of mistake that doesn't happen enough to annoy your colleagues when it catches it but is implicated in a recent outage management still remembers?

Then slowly adding rules over time as feasible (exploiting each outage as an opportunity to enlist stakeholder support for adding more rules that would have caught that particular misconfiguration event)

Sometimes I think figuring out how to stage and phase gradually making the changes you need to improve the system in light of social inertia against it is the most complex part of corporate enterprise software work. I definitely remember a time I wrote a whole perl-based wheel-reinventing crappy puppet knockoff just for a set of nagios plugins, entirely not for technical reasons, but for the political reason that this way we would control it instead of the other department which refused to get with the program and let us do staged canary rollouts. It was wrong technically if you assume frictionless corporate politics, but it was the right and only practical way to achieve the goal of ending the drumbeat of customer outages.

Re: A skeptic's first contact with Kubernetes

#14

Post by INTPenis »

Reminds me of myself, and probably many others on here. I've always been a skeptic of novelty. Just like with cryptocoins I was there on the ground floor, I remember when Docker was launching. I remember trying to understand what it was, and I remember someone comparing it to MSI for Linux.

Unlike cryptocoins I didn't miss out on being a millionaire with containers. :(

I just avoided containers until 2019! So to me it was first containers, and then kubernetes.

That way I was already sold on the whole container image concept and namespaces in your OS, I had used FreeBSD jails in the early 2000s.

So when I understood k8s I realized it's literally just a container orchestrator. It might seem complicated but that's all to do with being able to run containers on a range of nodes instead of just one. And of course having an API to tie it all together.

Whether you project needs that or not is something you should definitely explore in depth before you set out and try to use it. Personally I prefer container hosts over k8s for most startup projects. I look forward to Talos' new podman package and being able to deploy a minimal podman container host with Talos, no k8s necessary.

Re: A skeptic's first contact with Kubernetes

#15

Post by cyberax »

My problem with K8s: the network abstraction layer just feels _wrong_.

It's an attempt to replicate the old model of "hard exterior, gooey interior" model of corporate networks.

I would very much prefer if K8s used public routable IPv6 for traffic delivery, and then simply provided an authenticated overlay on top of it.

Re: A skeptic's first contact with Kubernetes

#16

Post by Atreiden »

Great writeup on the core fundamentals, saved this to share with engineers who are new to k8s and need a quick primer.

Re: This piece -

> Given the Controller pattern, why isn't there support for "Cloud Native" architectures?

> I would like to have a ReplicaSet which scales the replicas based on some simple calculation for queue depth (eg: queue depth / 16 = # replicas)

> Defining interfaces for these types of events (queue depth, open connections, response latency) would be great

> Basically, Horizontal Pod Autoscaler but with sensors which are not just "CPU"

HPAs are actually still what you want here - you can configure HPAs to scale automatically based on custom metrics. If you run Prometheus (or a similar collector), you can define the metric you want (e.g. queue-depth) and the autoscaler will make scaling decisions with these in mind.

Resources:

https://kubernetes.io/docs/tasks/run-application/horizontal-...

https://learnk8s.io/autoscaling-apps-kubernetes

Re: A skeptic's first contact with Kubernetes

#17

Post by entropyie »

Atreiden wrote: post #16

Great writeup on the core fundamentals, saved this to share with engineers who are new to k8s and need a quick primer. Re: This piece - > Given the Controller pattern, why isn't there support for "Cloud Native" architectures? > I would like to have a ReplicaSet which scales the replicas based on some simple calculation for queue depth (eg: queue depth / 16 = # replicas) > Defining interfaces for these types of events…

See KEDA and Karpenter for advanced k8s acaling

Re: A skeptic's first contact with Kubernetes

#18

Post by IceWreck »

JojoFatsani wrote: post #5

The helm yaml thing really is annoying. Unfortunately it feels like helm is too firmly embedded to unseat at this point.

I've switched to kustomize for my homelab. Its built into kubectl (no extra tool needed) and its not really templating.

It doesn't fully replace helm for 3rd party tools or packaging but its a solid alternative for less complicated setups.

Re: A skeptic's first contact with Kubernetes

#19

Post by nrr »

INTPenis wrote: post #14

Reminds me of myself, and probably many others on here. I've always been a skeptic of novelty. Just like with cryptocoins I was there on the ground floor, I remember when Docker was launching. I remember trying to understand what it was, and I remember someone comparing it to MSI for Linux. Unlike cryptocoins I didn't miss out on being a millionaire with containers. :( I just avoided containers until 2019! So to me i…

"Whether you project needs that or not is something you should definitely explore in depth before you set out and try to use it." Unfortunately, as someone who's written these kinds of orchestrators a few times already (going back to jails on FreeBSD 4.8 and Xen domUs on NetBSD 2.0) and has a nose for where they're a good idea, I find this is overwhelmingly not the prevailing wisdom. I desperately wish it were.

Somewhere, the industry went all-in on Kubernetes, and I have zero desire to touch most of the resulting software stacks. We sold folks hard on complexity being necessary, and Kubernetes makes it very tempting to go ham with it.

Re: A skeptic's first contact with Kubernetes

#20

Post by zikduruqe »

entropyie wrote: post #17
Atreiden wrote: post #16

Great writeup on the core fundamentals, saved this to share with engineers who are new to k8s and need a quick primer. Re: This piece - > Given the Controller pattern, why isn't there support for "Cloud Native" architectures? > I would like to have a ReplicaSet which scales the replicas based on some simple calculation for queue depth (eg: queue depth / 16 = # replicas) > Defining interfaces for these types of events…

See KEDA and Karpenter for advanced k8s acaling

^ this right here. We used KEDA to query DynamoDB to look at a queue depth we wrote to a table. If number was X, then we would scale on it. Was pretty slick.
Post reply on HN