Live data from Hacker News

Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

petekeen.net

21–30 of 66 posts

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#21
post #2

Hey, author here. This is a piece about moving away from kubernetes and toward something that I can actually maintain as a solo person who has a life outside of k9s. It's not really intended to be "anti-kubernetes", more like "kubernetes really is too hard for my purposes". IMO the best change that I've made has been to give deterministic IPv6 addresses to every container and then using those for ingress. I'm curious…

Did you ever consider NixOS for your homelab?

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#23
post #2

Hey, author here. This is a piece about moving away from kubernetes and toward something that I can actually maintain as a solo person who has a life outside of k9s. It's not really intended to be "anti-kubernetes", more like "kubernetes really is too hard for my purposes". IMO the best change that I've made has been to give deterministic IPv6 addresses to every container and then using those for ingress. I'm curious…

For transparency, I work with K8s every day and run it with FluxCD as my homelab and have in various formats for a few years.

Before that though, I had a single computer (a NUC if memory serves) with systemd running docker containers. Dead simple.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#24

Lot of kubernetes hate here, which is surprising. I run a little 3 node cluster and besides the hardware issues I had (long story), it has been rock solid and dead easy to setup. Talos + longhorn + fluxcd (optional), is super nice. And everything beyond that is additive and just works within the ecosystem. If anything, it helped keep my stuff alive during all the hardware issues a lot longer. I think like 5-6 years a…

Did you already know kubernetes, or did you have to learn it for this setup?

I believe this is the key difference, because it doesn't make sense to learn such a complicated tool for a simple use case.

If you already know it sure, it may make sense. But you are removing a huge initial effort required otherwise.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#25
post #5

IMO, kubernetes is overkill for a small non-homogeneous home cluster. What I use and really recommend is using systemd +/- docker. It just becomes so darn simple. Do not go the compose route (that route is filled with sadness of the incomplete stacks because db container failed silently kind) - instead aim to decompose the compose files and write a separate systemd service file for each of them, you can then assign l…

> What I use and really recommend is using systemd +/- docker

Even better, systemd+podman (=quadlets).

Quadlets are great, you basically can declare a compose file as a systemd unit with all the good and bad that comes with that.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#26
I did the same move as you away from k8s to plain proxmox containers and VMs. Professionally i do work with k8s, and see the benefits of it (not always, but i see the use cases), but in my homelab it was consuming a lot of energy. Just dropping the whole k8s, made me save 1 kw energy when idle... I guess mainly because of the active API, shifting workload from different workloads and the whole machinery that happens behind the scenes..

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#27
post #3

I still don't know really what Kubernetes is for or why so many people outside specific environments are using it, but it's cool that you're using Ruby.

Kubernetes is good for two things. Zero downtime deployments and self-healing (where the looping state mechanism comes in). There are people who want k8s to handle every single operation that can run on a server, do not listen to those kinds of people they will lead you astray.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#28

I did the same move as you away from k8s to plain proxmox containers and VMs. Professionally i do work with k8s, and see the benefits of it (not always, but i see the use cases), but in my homelab it was consuming a lot of energy. Just dropping the whole k8s, made me save 1 kw energy when idle... I guess mainly because of the active API, shifting workload from different workloads and the whole machinery that happens…

On low power, mostly idle systems that's definitely a downside. K8s is always busy, probably largely because it's whole architecture is centered around reconciliation loops which contentiously compare the actual state to the desired state. It's not much but enough to prevent the CPU from entering its lowest power states.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#29

I did the same move as you away from k8s to plain proxmox containers and VMs. Professionally i do work with k8s, and see the benefits of it (not always, but i see the use cases), but in my homelab it was consuming a lot of energy. Just dropping the whole k8s, made me save 1 kw energy when idle... I guess mainly because of the active API, shifting workload from different workloads and the whole machinery that happens…

I would have assumed that observability components (Prometheus, etc) dwarf the control plain when it comes to activity. At least with not extremely dynamic workloads. And you mention that it was due to shifting workloads. If the demands are static, nothing should shift in my experience?

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#30
post #3

I still don't know really what Kubernetes is for or why so many people outside specific environments are using it, but it's cool that you're using Ruby.

Kubernetes is good for two things. Zero downtime deployments and self-healing (where the looping state mechanism comes in). There are people who want k8s to handle every single operation that can run on a server, do not listen to those kinds of people they will lead you astray.

It's the industry standard for running and shipping containerized applications, at least for the ones which are more complex and/or dynamic than an application container and maybe 1-2 databases. It provides a well defined and stable orchestration API which is also quite capable. It's even sometimes worthwhile to spin up a single node cluster for that reason. Does not mean it is perfect, is it sometimes a bit complex? yes, is it verbose? yes, could the established ecosystem be better (thinking about e.g. helm)? yes. But at some point it's still easier with k8s than without it.

It is of course not the best choice for everything, there are enough things where you better just spin up a VM and deploy directly or with only a simple container runtime if you use containers.

Post reply on HN