> Kubernetes is Too Hard. I built a system that I didn't actually know how to maintain without the time or energy necessary to dig myself out of trouble. Couldn't agree more. Unless your homelab's point is to learn Kubernetes, just keep it simple. Proxmox sounds good, or just QEMU, libvirt, lxc, Docker, podman, whatever. Install packages, not containers where possible. Shell scripts are fine where needed. If it works…
Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
31–40 of 66 posts
Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#32IMO, kubernetes is overkill for a small non-homogeneous home cluster. What I use and really recommend is using systemd +/- docker. It just becomes so darn simple. Do not go the compose route (that route is filled with sadness of the incomplete stacks because db container failed silently kind) - instead aim to decompose the compose files and write a separate systemd service file for each of them, you can then assign l…
Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#33Lot of kubernetes hate here, which is surprising. I run a little 3 node cluster and besides the hardware issues I had (long story), it has been rock solid and dead easy to setup. Talos + longhorn + fluxcd (optional), is super nice. And everything beyond that is additive and just works within the ecosystem. If anything, it helped keep my stuff alive during all the hardware issues a lot longer. I think like 5-6 years a…
Are you running a startup at home.
Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#34I did the same move as you away from k8s to plain proxmox containers and VMs. Professionally i do work with k8s, and see the benefits of it (not always, but i see the use cases), but in my homelab it was consuming a lot of energy. Just dropping the whole k8s, made me save 1 kw energy when idle... I guess mainly because of the active API, shifting workload from different workloads and the whole machinery that happens…
Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#35Earlier quoted context omitted.
Somehow we were doing that with deployment scripts and VM management tooling before Kubernetes became a thing, and without having to deal with YAML spaghetti.
Yes, and Kubernetes came around as another player in that ecosystem and became popular for a reason, largely so we didn't have to manage clusters with imperative non-idempotent scripts with no runtime introspection or self-healing. I've done light devops (lab scale, not enterprise) off and on since cfengine was a thing, and while I'm no fan of the explosion of YAML (there's a special place in hell for helm in particu…
Indeed. Using text templating for structured documents is already quite bad, doing it with a significant whitespace language it's absolute hell.
Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#36Lot of kubernetes hate here, which is surprising. I run a little 3 node cluster and besides the hardware issues I had (long story), it has been rock solid and dead easy to setup. Talos + longhorn + fluxcd (optional), is super nice. And everything beyond that is additive and just works within the ecosystem. If anything, it helped keep my stuff alive during all the hardware issues a lot longer. I think like 5-6 years a…
That would not match their reason for preferring to move off k8:
> I built a system that I didn't actually know how to maintain without the time or energy necessary to dig myself out of trouble.
If you want a system that you can understand all at once in your own head then making it with your own head is a better way to go IMO, for a small system at least.
Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#37Earlier quoted context omitted.
Somehow we were doing that with deployment scripts and VM management tooling before Kubernetes became a thing, and without having to deal with YAML spaghetti.
Yes but Kubernetes takes these battle-tested scripts, and allow everyone to use them with a few lines of YAML ;) I understand the dislike of YAML but a Kubernetes deployment is ~50 lines, if I had to build my own scripts with a similar feature set I don't think I would be able to get it down much more than that.
Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#38Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#39Re: Rebuilding My Homelab with Compose, Ruby, IPv6, and No Kubernetes
#40I did the same move as you away from k8s to plain proxmox containers and VMs. Professionally i do work with k8s, and see the benefits of it (not always, but i see the use cases), but in my homelab it was consuming a lot of energy. Just dropping the whole k8s, made me save 1 kw energy when idle... I guess mainly because of the active API, shifting workload from different workloads and the whole machinery that happens…
Proxmox is merely Debian with some fancy curtains. I'm sort of fascinated by this social belief it is something more than that. Although I agree with your sentiment, I'd rather be on XCP-ng or IncusOS as the foundation for containers. I don't believe sacrificing immutability in the host is an improvement over k8s for stability and maintenance.
It's part of the appeal that it is mostly just Debian under the hood. If I want to run containers I would also not see it as the best choice but it really shines in managing qemu VMs.