Awesome. Gonna give this a go. Recently acquired a 3rd Rasp4 to have a quorum of fast-ish ones.
Concerned about the ARM aspect though. On docker at least the ecosystem felt a lot smaller on ARM.
Raspberry Pi Homelab with Kubernetes
21–30 of 52 posts
Re: Raspberry Pi Homelab with Kubernetes
#22"so that you have something pretty to show your non-technically inclined significant other as the output of your hard work"
Gosh, that resonates... :)
Also very excited to read part 2!
Re: Raspberry Pi Homelab with Kubernetes
#23I spent some time getting pi-hole to run on a raspi k3s cluster last year and wrote about it. Hopefully there's something useful in my investigation for your current project.
https://medium.com/@subtlepseudonym/pi-hole-on-kubernetes-87...
Re: Raspberry Pi Homelab with Kubernetes
#24As has become my habit lately, I'll chime in and say if you're interested in accessing your self-hosted services from the internet, IMO tunneling is the way to go: https://github.com/anderspitman/awesome-tunneling
Most of my services are only available by using a vpn into my home network. I can understand why you might need a public facing service but I avoid it like the plague.
Re: Raspberry Pi Homelab with Kubernetes
#25I have what I feel is an irrational desire to build an RPi k8s cluster. I feel like I would be better served by a small x86 box, but something about having a real cluster really appeals to me.
I really tried! But maybe because my setup was a bit demanding, rPi4 couldn't handle it well. (home-assistant, adguard home, nextcloud, plex, *arr apps, and many more on docker) I just had to switch to a SFF desktop PC with Ubuntu on it and never looked back. For toying around rPis can be fun but in real life applications, they were getting extremely hot. Had to add extra fans etc. I still use my rPi4s, one of them c…
Why kind of quality did you get running plex? I doubt a cluster of pi’s could keep up with the transcoding.
Re: Raspberry Pi Homelab with Kubernetes
#26As has become my habit lately, I'll chime in and say if you're interested in accessing your self-hosted services from the internet, IMO tunneling is the way to go: https://github.com/anderspitman/awesome-tunneling
Most of my services are only available by using a vpn into my home network. I can understand why you might need a public facing service but I avoid it like the plague.
I see it as an important step to a more decentralized future. For example, I know a few people who maintain Plex servers for their friends and family. This works quite well, but getting the server on the internet is the trickiest part. You can sink a lot of time into configuring routers, managing certs, NAT, DMZ, LMNOP. Or you can use a tunneling service that manages all of it for you.
Re: Raspberry Pi Homelab with Kubernetes
#27I have what I feel is an irrational desire to build an RPi k8s cluster. I feel like I would be better served by a small x86 box, but something about having a real cluster really appeals to me.
I know that feeling! Pulled the trigger on that three times now. My first cluster was Raspis and ODroids. The two issues I had - not everything runs on Arm (yet) and memory constraints were tough to work with. My second cluster - 4 Atomic Pis. The (Intel Atom) quad cores were fine but the 2GB of memory was a real problem when I started playing with more interesting workloads. The current "production" home cluster is…
This sounds like a neat setup - i used to run a set of 3 old compuets, a Nuc, and old laptops. But they were quite low spec, so i sold them and bought a small htpc instead, quadcore, with 16 gigs of ram and 512Gb nvme.
Also using RKE, it looks like the best cluster 'distro' around.
I used longhorn but found it a little slow, putting it on SSD was a bit of a waste. I tried ceph, it was much faster but i just do not understand how it works, etc. Well enoguh to fix anything if the cluster goes wrong.
Re: Raspberry Pi Homelab with Kubernetes
#28I have what I feel is an irrational desire to build an RPi k8s cluster. I feel like I would be better served by a small x86 box, but something about having a real cluster really appeals to me.
Been there, dropped 200+€ on rpis only to find out I then had to spend almost the same amount on microstld cards, power supplies, cables and all the other stuff.
Sold all those, recovered most of the money and bought a dismissed laptop from my employer of the time: 3rd gen quad-core i7, 16gb ram (later upgraded to 32), 480gb ssd + 750gb hdd for waaaay less (just the residual price)
Installed proxmox, created small VMS and played with k8s and a lot more stuff in a cheaper and more performant way (full gigabit ethernet when RPI has usb-shared ethernet, ssd-grade I/O performance, core i7 compute performance).
If you want to experiment the raspberry pi is the dumbest thing that you can buy.
Go for old hardware and virtualization, you'll also learn more (containers are here to stay, but VMs aren't going away either).
Power usage was also surely higher than a single RPI, but negligible anyway: laptop Intel processors with speedstep technology that lower down power absorption, and can go down to like 15-20 watts/hour.
Re: Raspberry Pi Homelab with Kubernetes
#29I've been looking into doing an SBC K8s cluster, but had my eyes on the ODroid N2+. Its big.LITTLE design with 4 high-performance cores and 2 low-performance ones would not only boost the compute capacity vs the RPi4, but also perhaps cgroups on the nodes could be configured such that pods only run on the 4 high-performance cores, leaving the 2 low-performance ones available for system daemons, kubelet, etc. The bigg…
I've been using Rock64 boards (4 gig RAM models.) There is plenty of free memory on the master nodes.
Re: Raspberry Pi Homelab with Kubernetes
#30I understand the allure of real hardware and doing this on a physical cluster but I'm always surprised not to see K8s/k3s/whatever running under (inside?) LXD as a learning/experimentation tool discussed more often.
The physical cabling, underlying operating system, bootstrapping etc strikes me as the least of the K8s learning experience and the only advantage of a toy running on Pis vs LXD. Not to mention most K8s deployments these days will be in some cloud provider of choice where most of this is handled... In that case LXD on a local machine with software network bridging, etc probably more closely approximates what most people will go to production with anyway.
I'm a little sour on the whole "LXD only officially distributed via snap" thing too but at least on Ubuntu 18.04 and forward getting a toy X node Kubernetes cluster up and running is trivial and costs nothing more than RAM and disk space. As is commonly known LXD doesn't even require a hypervisor so the hardware requirements are (essentially) anything x86_64. It's also fun to spin up/destroy any number of instances at will just using the command line.
MicroK8s even provides an LXD how-to that should work for your flavor of choice (with a little adaptation, of course):
Of course if you're actually doing some kind of edge deployment or whatever on actual Raspberry Pi/armv7/arm64 hardware that's the obvious way to go (or you can just run LXD on your Pi) :).