Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear friend, you have built a Kubernetes (2024)

macchaffee.com

Post reply on HN
Print view View original

81–90 of 183 posts

Re: Dear friend, you have built a Kubernetes (2024)

#81

Post by cassianoleal »

supriyo-biswas wrote: post #14
cassianoleal wrote: post #6

Earlier quoted context omitted.

I have been using Kubernetes for 7 or 8 years now, and have nearly 100% stayed away from Helm. Some Kustomize, a little bit of envsubst and we're good to go thank you very much.

How do you handle cleanups and hooks? The best way to do helm, at least for me, seems to be about limiting its use to simple templating use cases; if you end up needing an if, you've probably done something terribly wrong.

That's my main gripe with Helm.

For the simple use case you're describing, Helm is not required. Plenty of other solutions around.

For use cases where it starts getting useful, we both agree that something has gone terribly wrong.

I still don't know why Helm exists. It's a solution that created lots of problems that didn't exist.

Re: Dear friend, you have built a Kubernetes (2024)

#82

Post by cortesoft »

throwawaypath wrote: post #79
cortesoft wrote: post #76

Earlier quoted context omitted.

It is if you stick to standard Kubernetes resources, and it has gotten even easier with better storage class and load balancer support. All of the cloud providers now give you default storage classes and ingresses when you provision a cluster on them, so you can use the exact same deployment on any of them an automatically get those things provisioned in the right way out of the box.

>It is if you stick to standard Kubernetes resources "If you stick to standard C..." No one does, that's the issue. Helm charts that only support certain cloud providers, operators and annotations that end up being platform specific, etc. >now give you default storage classes and ingresses Ingress is being deprecated, it's Gateway now! Welcome to hell, er, Kubernetes.

I use Kubernetes every day, and have worked with dozens of helm charts, and have yet to encounter cloud specific helm charts. Are these internal helm charts for your company?

Obviously you can lock yourself in if you choose, but I have yet to see third party tools that assume a specific provider (unless you are using tools created BY that provider).

At my previous spot, we were running dozens of clusters, with some on prem and some in the cloud. It was easy to move workloads between the two, the only issue was ACLs, but that was our own choice.

I know they are pushing the new gateway api, but ingresses still work just fine.

Re: Dear friend, you have built a Kubernetes (2024)

#83

Post by subhobroto »

oddurmagnusson wrote: post #66
subhobroto wrote: post #42

Earlier quoted context omitted.

> found myself rolling out Yoink - using Tailscale SSH is brilliant - using caddy-docker-proxy for ingress is brilliant What do you use for: - service discovery - secret store (EDIT: Crap you use Infisical. No shade, I just have this horrible foreboding it will end up like Hashicorp. I use Conjur Secretless Broker but am tracking: https://news.ycombinator.com/item?id=47903690 ) - backing up and restoring state like i…

Service discovery is basically just Docker's internal DNS. Caddy-docker-proxy can use it to find healthy upstreams. For secrets, I self-host Infisical on the box -- easy to plug in whatever secret manager, should make it pair nicely with https://github.com/tellerops/teller or something similar Had no problems with Hertzner so far, just enjoying the raw CPU power of bare metal. The plan is to roll out more boxes acros…

I have a suspicion you're using Headscale? If so, I urge you to consider Ionscale. I use it with Authentik as the IdP.

Personally commiting to using Tailscale as a core foundation of my infrastructure and Ionscale is my hedge against getting Hashicorped.

> Service discovery is basically just Docker's internal DNS. Caddy-docker-proxy can use it to find healthy upstreams

Do you have a writeup of this somewhere? I'm unaware of being able to manage Docker's internal DNS over some kind of an API (would appreciate if you know a way to). The only way I know is to manipulate network aliases via Docker Engine API. As a result I use Hickory DNS with RFC 2136. That coupled with Caddy-docker-proxy gets me extremely close.

Re: Dear friend, you have built a Kubernetes (2024)

#84

Post by subhobroto »

stavros wrote: post #63
subhobroto wrote: post #59

Earlier quoted context omitted.

I am a big fan which is why I am saying this: you're dismissing the kernel and ABI surface is a huge assumption that must hold true for your comment to hold stavros. If you had said "unikernels" I would have had no arguments to make.

What do you mean? Statically linked programs depend on the kernel too.

right - that's precisely what I meant. I read your comment "Containers are just statically-linked programs for the rest of us." as "containers can be replaced by statically-linked programs".

If you didn't imply that, I apologize.

If you did mean that, I disagree with you precisely because your point works if you only care about dependency management - it falls apart on system state. A static binary is a process on the host and shares the same process space, network stack, and filesystem.

OTOH, a container is a jail (the primary usecase): I can't cgroup a static binary's memory usage or give it a virtual network interface without reimplementing "container lite". Containers aren't just 'statically linked programs' - they allow me to use the kernel as a hypervisor for isolated environments.

What they are though, a messy but practical compromise to Unikernels - which was my last point in our GP.

Re: Dear friend, you have built a Kubernetes (2024)

#85

Post by stavros »

subhobroto wrote: post #84
stavros wrote: post #63

Earlier quoted context omitted.

What do you mean? Statically linked programs depend on the kernel too.

right - that's precisely what I meant. I read your comment "Containers are just statically-linked programs for the rest of us." as "containers can be replaced by statically-linked programs". If you didn't imply that, I apologize. If you did mean that, I disagree with you precisely because your point works if you only care about dependency management - it falls apart on system state. A static binary is a process on th…

Oh, no, I meant that containers serve the same purpose as statically linked programs for languages that can't do that. Eg if you want to deploy a Python codebase, a container is a good way to include all dependencies.

I didn't mean "containers don't have any advantages compared to statically linked programs".

Re: Dear friend, you have built a Kubernetes (2024)

#87

Post by drdaeman »

They have built an orchestrator, not Kubernetes. There is one key difference: they know this thing, end-to-end, down to every single bolt and piece of duct tape (with possible exception for Docker internals)

And that's a very important distinction when it comes to maintaining complex systems. This could've changed with LLMs (I'm still adjusting to what new capabilities mean for various decision-making logic), but before machine intelligence debugging an issue with Kubernetes could've been a whole world of pain.

Re: Dear friend, you have built a Kubernetes (2024)

#88

Post by vbezhenar »

cortesoft wrote: post #11

This is obviously slightly exaggerated, but I do feel like this whenever people dismiss Kubernetes as either too complicated or not needed. The response I always got when suggesting Kubernetes is "you can do all those things without Kubernetes" Sure, of course. There are a million different ways to do everything Kubernetes does, and some of them might be simpler or fit your use case more perfectly. You can make diffe…

Yeah, I spent quite a bit of time learning Kubernetes, but now I'd use it to host a static webpage on a single server, over alternatives. It's so awesome.

Re: Dear friend, you have built a Kubernetes (2024)

#89

Post by vbezhenar »

throwawaypath wrote: post #79
cortesoft wrote: post #76

Earlier quoted context omitted.

It is if you stick to standard Kubernetes resources, and it has gotten even easier with better storage class and load balancer support. All of the cloud providers now give you default storage classes and ingresses when you provision a cluster on them, so you can use the exact same deployment on any of them an automatically get those things provisioned in the right way out of the box.

>It is if you stick to standard Kubernetes resources "If you stick to standard C..." No one does, that's the issue. Helm charts that only support certain cloud providers, operators and annotations that end up being platform specific, etc. >now give you default storage classes and ingresses Ingress is being deprecated, it's Gateway now! Welcome to hell, er, Kubernetes.

> Ingress is being deprecated

Do you have any links about Ingress being deprecated?

Official docs here: https://kubernetes.io/docs/reference/kubernetes-api/service-...

There are no mentions about this API being deprecated.

Re: Dear friend, you have built a Kubernetes (2024)

#90

Post by throwaway041207 »

shrubble wrote: post #58

I can tell you how vendors deliver a software solution that runs on Kubernetes: very poorly. The needed tweaks, the ability to customize things, basically goes to zero because the support staff is technical about the software, but NOT about Kubernetes. I am not joking: a recent deployment required 3x VMs for Kubernetes, each VM having 256 gigabytes of RAM; then a separate 3x VMs for a different piece. 1.5TB of RAM to…

Whether this is deployed via Helm charts or a native controller, there's almost certainly some overlay where you can override resource values, unless this is just a very crappy vendor.
Post reply on HN