Live data from Hacker News

Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

petekeen.net

31–40 of 66 posts

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#31
post #6

> Kubernetes is Too Hard. I built a system that I didn't actually know how to maintain without the time or energy necessary to dig myself out of trouble. Couldn't agree more. Unless your homelab's point is to learn Kubernetes, just keep it simple. Proxmox sounds good, or just QEMU, libvirt, lxc, Docker, podman, whatever. Install packages, not containers where possible. Shell scripts are fine where needed. If it works…

I enjoy using Kubernetes on bare metal nodes at home and took it over Proxmox, quite happy with it (not so happy with costs of hardware to expand the cluster though)

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#32
post #5

IMO, kubernetes is overkill for a small non-homogeneous home cluster. What I use and really recommend is using systemd +/- docker. It just becomes so darn simple. Do not go the compose route (that route is filled with sadness of the incomplete stacks because db container failed silently kind) - instead aim to decompose the compose files and write a separate systemd service file for each of them, you can then assign l…

I disagree - I think k8s is absolutely the right choice at that level. We use ECS instead of k8s for small projects because the control plane is more than the actual service, but when you want container management, some sort of basic deployment management, service discovery, storage management and secret management, k8s is super simple to run and works pretty much out of the box. For a homelab, I'd expect you to want this. Building it all yourself may be fun (and that's a good reason to do it), but it's going to result in all the complexity with a lot of issues that k8s just solves for you.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#33

Lot of kubernetes hate here, which is surprising. I run a little 3 node cluster and besides the hardware issues I had (long story), it has been rock solid and dead easy to setup. Talos + longhorn + fluxcd (optional), is super nice. And everything beyond that is additive and just works within the ecosystem. If anything, it helped keep my stuff alive during all the hardware issues a lot longer. I think like 5-6 years a…

> Talos + longhorn + fluxcd

Are you running a startup at home.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#34

I did the same move as you away from k8s to plain proxmox containers and VMs. Professionally i do work with k8s, and see the benefits of it (not always, but i see the use cases), but in my homelab it was consuming a lot of energy. Just dropping the whole k8s, made me save 1 kw energy when idle... I guess mainly because of the active API, shifting workload from different workloads and the whole machinery that happens…

Proxmox is merely Debian with some fancy curtains. I'm sort of fascinated by this social belief it is something more than that. Although I agree with your sentiment, I'd rather be on XCP-ng or IncusOS as the foundation for containers. I don't believe sacrificing immutability in the host is an improvement over k8s for stability and maintenance.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#35
post #15

Earlier quoted context omitted.

Somehow we were doing that with deployment scripts and VM management tooling before Kubernetes became a thing, and without having to deal with YAML spaghetti.

Yes, and Kubernetes came around as another player in that ecosystem and became popular for a reason, largely so we didn't have to manage clusters with imperative non-idempotent scripts with no runtime introspection or self-healing. I've done light devops (lab scale, not enterprise) off and on since cfengine was a thing, and while I'm no fan of the explosion of YAML (there's a special place in hell for helm in particu…

> there's a special place in hell for helm in particular for using text/template to generate yaml

Indeed. Using text templating for structured documents is already quite bad, doing it with a significant whitespace language it's absolute hell.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#36

Lot of kubernetes hate here, which is surprising. I run a little 3 node cluster and besides the hardware issues I had (long story), it has been rock solid and dead easy to setup. Talos + longhorn + fluxcd (optional), is super nice. And everything beyond that is additive and just works within the ecosystem. If anything, it helped keep my stuff alive during all the hardware issues a lot longer. I think like 5-6 years a…

> an LLM can probably

That would not match their reason for preferring to move off k8:

> I built a system that I didn't actually know how to maintain without the time or energy necessary to dig myself out of trouble.

If you want a system that you can understand all at once in your own head then making it with your own head is a better way to go IMO, for a small system at least.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#37
post #15

Earlier quoted context omitted.

Somehow we were doing that with deployment scripts and VM management tooling before Kubernetes became a thing, and without having to deal with YAML spaghetti.

Yes but Kubernetes takes these battle-tested scripts, and allow everyone to use them with a few lines of YAML ;) I understand the dislike of YAML but a Kubernetes deployment is ~50 lines, if I had to build my own scripts with a similar feature set I don't think I would be able to get it down much more than that.

Also your script often might not be that battle tested, more likely it will have bugs and miss a few important edge cases.

Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes

#40

I did the same move as you away from k8s to plain proxmox containers and VMs. Professionally i do work with k8s, and see the benefits of it (not always, but i see the use cases), but in my homelab it was consuming a lot of energy. Just dropping the whole k8s, made me save 1 kw energy when idle... I guess mainly because of the active API, shifting workload from different workloads and the whole machinery that happens…

Proxmox is merely Debian with some fancy curtains. I'm sort of fascinated by this social belief it is something more than that. Although I agree with your sentiment, I'd rather be on XCP-ng or IncusOS as the foundation for containers. I don't believe sacrificing immutability in the host is an improvement over k8s for stability and maintenance.

> Proxmox is merely Debian with some fancy curtains.

It's part of the appeal that it is mostly just Debian under the hood. If I want to run containers I would also not see it as the best choice but it really shines in managing qemu VMs.

Post reply on HN