Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear friend, you have built a Kubernetes (2024)

macchaffee.com

Post reply on HN
Print view View original

61–70 of 183 posts

Re: Dear friend, you have built a Kubernetes (2024)

#61

Post by subhobroto »

antonvs wrote: post #47
zdw wrote: post #2

IMO, Kubernetes isn't inevitable, and this seems to paint it as such. K8s is well suited to dynamically scaling a SaaS product delivered over the web. When you get outside this scenario - for example, on-prem or single node "clusters" that are running K8s just for API compatibility, it seems like either overkill or a bad choice. Even when cloud deployed, K8s mostly functions as a batteries-not-included wrapper around…

Kubernetes, in the form of k3s, was a critical success factor for us with the onprem deployment of our SaaS product. What's the problem with a single-node cluster? We use that for e.g. dev environments, as well as some small onprem deployments. > Even when cloud deployed, K8s mostly functions as a batteries-not-included wrapper around the underlying cloud provider services and APIs. Which batteries are not included?…

> Kubernetes, in the form of k3s, was a critical success factor for us with the onprem deployment of our SaaS product.

What surprises and gotchas did you have to deal with using k3s as a Kubernetes implementation?

Did you use an LB? Which one? I'm assuming all your onprem nodes were just linux servers with very basic equipment (the fanciest networking equipment you used were 10GbE PCIe cards, nothing more special than that?)

Re: Dear friend, you have built a Kubernetes (2024)

#62

Post by kirici »

isodev wrote: post #3

Unless you’re in Erlang world (Elixir, Gleam..) and all that is already baked into OTP and the BEAM. You can go on holiday knowing it will be a while longer before you need to break out the pods (and at that scale, you will be able to afford a colleague or two to help you).

How does Elixir renew my certificates, mount networked storage mounts and create a VIP and internal DNS entries for my valkey instance?

Scaling is a sidenote, that it becomes easy is a result of hoisting everything else onto one control plane and a set of coherent APIs.

Re: Dear friend, you have built a Kubernetes (2024)

#63

Post by stavros »

subhobroto wrote: post #59
stavros wrote: post #21

Earlier quoted context omitted.

Containers are just statically-linked programs for the rest of us.

I am a big fan which is why I am saying this: you're dismissing the kernel and ABI surface is a huge assumption that must hold true for your comment to hold stavros. If you had said "unikernels" I would have had no arguments to make.

What do you mean? Statically linked programs depend on the kernel too.

Re: Dear friend, you have built a Kubernetes (2024)

#64

Post by baby_souffle »

cortesoft wrote: post #11

This is obviously slightly exaggerated, but I do feel like this whenever people dismiss Kubernetes as either too complicated or not needed. The response I always got when suggesting Kubernetes is "you can do all those things without Kubernetes" Sure, of course. There are a million different ways to do everything Kubernetes does, and some of them might be simpler or fit your use case more perfectly. You can make diffe…

> However, the big win with Kubernetes is that all of those choices have been made and agreed upon, and now you have an entire ecosystem of tools, expertise, blog posts, AI knowledge, etc, that knows the choices Kubernetes made and can interface with that. This is VERY powerful.

Yep! I am now using k8s even for small / 'single purpose' clusters just so I can keep renovate/argo/flux in the loop. Yes, I _could_ wire renovate up to some variables in a salt state or chef cookbook and merge that to `main` and then have the chef agent / salt minion pick up the new version(s) and roll them out gradually... but I don't need to, now!

Re: Dear friend, you have built a Kubernetes (2024)

#65

Post by kube-system »

Kubernetes is a powerful tool for complicated problems. If if it seems complicated, you probably don’t have a complicated deployment problem.

But really this applies to any powerful tool. If you need to measure a voltage, an 4 channel oscilloscope also probably seems too complicated.

Re: Dear friend, you have built a Kubernetes (2024)

#66

Post by oddurmagnusson »

subhobroto wrote: post #42
oddurmagnusson wrote: post #7

Found this the same day I published this: https://github.com/oddur/yoink Kubernetes was overkill (I do that all day, 5 days a week); Kamal was too restrictive, so I found myself rolling out Yoink. Just what I need from k8s, but simple enough I can point it to a baremetal machine on Hertzner that can easily run all my workloads.

> found myself rolling out Yoink - using Tailscale SSH is brilliant - using caddy-docker-proxy for ingress is brilliant What do you use for: - service discovery - secret store (EDIT: Crap you use Infisical. No shade, I just have this horrible foreboding it will end up like Hashicorp. I use Conjur Secretless Broker but am tracking: https://news.ycombinator.com/item?id=47903690 ) - backing up and restoring state like i…

Service discovery is basically just Docker's internal DNS. Caddy-docker-proxy can use it to find healthy upstreams.

For secrets, I self-host Infisical on the box -- easy to plug in whatever secret manager, should make it pair nicely with https://github.com/tellerops/teller or something similar

Had no problems with Hertzner so far, just enjoying the raw CPU power of bare metal. The plan is to roll out more boxes across different providers, using Tailscale for the backplane network and Cloudflare to load-balance between them. All in due time What issues have you been having ?

Re: Dear friend, you have built a Kubernetes (2024)

#67

Post by winton »

zbentley wrote: post #55
winton wrote: post #16

"Except if you quit or go on vacation, who will maintain this custom pile of shell scripts?" LLMs can reason about and fix them quite well.

Not as well as they can reason (or others can google) something as standardized as kubernetes. There’s just less context (in both senses of the term) needed to understand something running on a common substrate versus something bespoke, even if the bespoke thing is itself comprised of standardized parts.

For a project set up by a qualified engineer, there would be little difference to the end user in practice. The LLM would work out a solution with a negligible difference in speed. Maybe debugging would also be faster for the LLM without the abstraction layers and low level access?

Re: Dear friend, you have built a Kubernetes (2024)

#68

Post by antonvs »

subhobroto wrote: post #61
antonvs wrote: post #47

Earlier quoted context omitted.

Kubernetes, in the form of k3s, was a critical success factor for us with the onprem deployment of our SaaS product. What's the problem with a single-node cluster? We use that for e.g. dev environments, as well as some small onprem deployments. > Even when cloud deployed, K8s mostly functions as a batteries-not-included wrapper around the underlying cloud provider services and APIs. Which batteries are not included?…

> Kubernetes, in the form of k3s, was a critical success factor for us with the onprem deployment of our SaaS product. What surprises and gotchas did you have to deal with using k3s as a Kubernetes implementation? Did you use an LB? Which one? I'm assuming all your onprem nodes were just linux servers with very basic equipment (the fanciest networking equipment you used were 10GbE PCIe cards, nothing more special tha…

We sell to enterprise customers. All of them deploy our solution on internal cloud-style VM clusters. We use the Traefik ingress controller by default.

There really weren't any particular surprises or gotchas at that level.

In this context, I've never had to deal with anything at the level of the type of Ethernet card. That's kind of the point: platforms like k8s abstract away from that.

Re: Dear friend, you have built a Kubernetes (2024)

#69

Post by PufPufPuf »

blindlobstar wrote: post #43

Why both posts mention docker compose and not mentioning docker swarm. Being using it for my projects for long time. And it's so nice. Similar syntax, easy networking, rollout strategy, easy to add nodes to cluster. You can have one template docker-compose.yaml file and separate deployment files for different envs, like: docker-compose.dev.yaml, docker-compose.prod.yaml I think swarm is really underrated

I've been there. We still ended up with messy deploy scripts written in Ruby and the only debugging solution was "just comment out everything then run line by line".

Re: Dear friend, you have built a Kubernetes (2024)

#70

Post by arkeros »

supriyo-biswas wrote: post #14
cassianoleal wrote: post #6

Earlier quoted context omitted.

I have been using Kubernetes for 7 or 8 years now, and have nearly 100% stayed away from Helm. Some Kustomize, a little bit of envsubst and we're good to go thank you very much.

How do you handle cleanups and hooks? The best way to do helm, at least for me, seems to be about limiting its use to simple templating use cases; if you end up needing an if, you've probably done something terribly wrong.

You can rely purely on kubectl with something like:

cat manifests.yaml | kubectl apply -f - --server-side --field-manager "$FIELD_MANAGER" --prune --applyset "$APPLYSET" --namespace "$NAMESPACE"

Post reply on HN