Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

jeffgeerling.com

Post reply on HN
Print view View original

51–60 of 80 posts

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#51

Post by TheOperator »

gatherhunterer wrote: post #13
tyingq wrote: post #5

Thankfully these days, places like Digital Ocean and Linode have managed K8s where you only pay for the compute nodes, for $5/month each. So it's fairly cheap and easy to learn on a "real" cluster without having to build one.

Building a cluster is a fun and relatively easy project. You learn much more by having the hardware at your fingertips. You can simulate network failures and power failures or you can crash an important daemon. By causing problems you can see how K8s responds and manages itself when it does not have the nodes it expects. It is important to know these things because, for example, if you create a pod instead of a repli…

I bought into the rhetoric about the superiority of using real tech when I was studying for IT. It will make you more competent when you actually get on the job. It gets you used to encountering issues that come up in real environments far more than virtualized/simulated ones E.G. loose cables.

Yet to be honest I wouldn't recommend going the real tech route because I don't think it justifies the time and money investment when you can learn the lessons it teaches you on the job. One thing I realised is that it really doesn't really help you pass certs or look good in interviews aside from your ability to say "I made an k8s cluster out of raspberry pis". I might endorse it from a strict fun standpoint.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#52

Post by bndw »

lhuser123 wrote: post #38
bndw wrote: post #29

Earlier quoted context omitted.

Kind is explicitly not for production workloads[0]: Non-Goals: Being “production workload ready” - kind is meant to be used: for testing Kubernetes itself for testing against Kubernetes (EG in CI on Travis, Circle, etc.) for “local” clusters on developer machines NOT to host workloads serving user traffic etc. K3s seems happy to support production workloads. It's a single binary and gets you a cluster in one command:…

What are the benefits you find in using Kubernetes for a single-node cluster on a vps?

I host a few personal websites and Kubernetes allows me abstract the server setup. I don't need a highly available cluster and found k3s on a single node to be a nice replacement for manually configuring a reverse proxy, systemd services, deployment scripts, etc.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#53

Post by darren0 »

mkesper wrote: post #8
raesene9 wrote: post #6

A good method for learning how clusters work and playing with them without having to spend a lot of time re-building when you break things is kind ( https://kind.sigs.k8s.io/ ). Each node is a Docker container, but the version of Kubernetes running inside it is vanilla Kubeadm, so it's quite representative of what "real" clusters would look like. The great thing about it is you can spin up a cluster in < 2 mins local…

How does this compare to k3s? https://rancher.com/docs/k3s/latest/en/

Disclaimer: k3s creator

Kind is intended to be a development tool to run fully upstream clusters using kubeadm in Docker containers.

k3s is a custom distro that is oriented towards lightweight environments. The scope of k3s and kind are different (distro vs tool to run k8s in docker). There is another project called k3d (https://github.com/rancher/k3d) that more directly compares to kind. k3d will run k3s clusters in Docker. k3d creates clusters significantly faster and uses less mem/cpu resources than kind.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#54

Post by darren0 »

moondev wrote: post #30
mkesper wrote: post #8

Earlier quoted context omitted.

How does this compare to k3s? https://rancher.com/docs/k3s/latest/en/

KIND uses kubeadm which is the upstream standard for bootstrapping. It also supports multi masters and even launches a haproxy load balancer for kube-apiserver! K3S uses it's own bootstrapping, as well as non upstream binaries. It also replaces etcd with it's own shim. Non-kubeadm bootstrapped clusters are snowflakes in my opinion. You will find many docs that will not work with k3s due to missing systemd. You also l…

k3s has no direct relationship to systemd, I'm not sure what things not working you are referring too.

k3s has full flexibility to change any k8s option or component, it merely defaults to settings to meet it's goals of being a simple, lightweight, and secure distro.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#55

Post by darren0 »

raesene9 wrote: post #18
mkesper wrote: post #8

Earlier quoted context omitted.

How does this compare to k3s? https://rancher.com/docs/k3s/latest/en/

k3s is usually (AFAIK) run on VMs for production purposes, rather than being used for testing, which is where kind shines. The major advantage of kind is you download a single binary, run one command and quickly have a functional, disposable cluster, anywhere that Docker runs.

k3s also shines in dev/test/CI. It's smaller and spins up faster and has specific features to preload applications on startup. It's a single ~50mb binary also so it's fast to download and run in CI. Many users have moved from kind to k3s in CI because of its speed, flexibility, and simplicity.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#56

Post by bogomipz »

moondev wrote: post #36
bogomipz wrote: post #34

Earlier quoted context omitted.

Does Rook give you the equivalent of EBS root volumes for your nodes then? Is that the function you have it providing? Does it offer something beyond using local host storage and minio? I ask because I've generally been confused about the use case for Rook despite having read the "what is Rook?" paragraph many times on the project home page. My assumption is that it lets you build your own internal cloud provider. Is…

Not directly, rook provides a storage backend for in-cluster workload persistent volumes. In theory you could manage kvm machines with kubevirt and back the machines with PVCs from rook. I have not tried this and would be curious how the performance is.

>"Not directly, rook provides a storage backend for in-cluster workload persistent volumes."

Right so is it fair to say that the use case for Rook is if you are running Kubernetes on bare metal? For instance if I'm Kubernetes cluster on AWS then AWS already provides PVC via EBS and S3 volumes. Or am I overlooking a use case where you would run Rook on cluster running on AWS/GCP?

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#57

Post by ljm »

I suppose I was lucky dabbling in one of my side-projects because I had money to burn. I put it all in Google Cloud and then learned just how much you have to complicate the stack (beyond the complication of K8S) to lower infra costs.

Suddenly I wasn't using basic Kubernetes any more, I was setting up a new ingress controller so Google wouldn't launch a new load balancer instance for every public service I exposed. Those things aren't cheap.

It was an amazing way to experience just how much you can suffer in the cloud, and just how far you can go down the rabbit hole with this kind of tech.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#58

Post by naniwaduni »

bndw wrote: post #29
mkesper wrote: post #8

Earlier quoted context omitted.

How does this compare to k3s? https://rancher.com/docs/k3s/latest/en/

Kind is explicitly not for production workloads[0]: Non-Goals: Being “production workload ready” - kind is meant to be used: for testing Kubernetes itself for testing against Kubernetes (EG in CI on Travis, Circle, etc.) for “local” clusters on developer machines NOT to host workloads serving user traffic etc. K3s seems happy to support production workloads. It's a single binary and gets you a cluster in one command:…

"not meant to be used" mostly just means that when something breaks, you're on your own.

Which, you know, you were anyway.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#59

Post by JeremyNT »

jptoto wrote: post #7

I LOVE Jeff's work and I own his Ansible book. The setup is pretty awesome. FWIW I think you could do this with local VMs a little more easily and provision with Vagrant. Having said that, a cluster of Pis is super fun!

Yeah, using local vms is going to greatly expand access to this kind of learning. A current developer laptop is more than beefy enough, and I'd suggest that there aren't many useful lessons to learn from the pi hardware that can't be learned fully virtualized.

But if you're having fun, of course, more power to you :)

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#60

Post by moondev »

bogomipz wrote: post #56
moondev wrote: post #36

Earlier quoted context omitted.

Not directly, rook provides a storage backend for in-cluster workload persistent volumes. In theory you could manage kvm machines with kubevirt and back the machines with PVCs from rook. I have not tried this and would be curious how the performance is.

>"Not directly, rook provides a storage backend for in-cluster workload persistent volumes." Right so is it fair to say that the use case for Rook is if you are running Kubernetes on bare metal? For instance if I'm Kubernetes cluster on AWS then AWS already provides PVC via EBS and S3 volumes. Or am I overlooking a use case where you would run Rook on cluster running on AWS/GCP?

Yes, rook is great for bare metal and would enable dynamically provisioned persistent volumes.

Rook runs a ceph cluster inside your kubernetes cluster to provide the storage. The downside is this consumes cpu/memory (and obviously storage) resources to run, whereas on AWS, EBS is integrated into the platform so it does not "run" inside your cluster (other than the aws cloud-provider that provides the integration).

If you wanted to run the same storage backend on bare metal and AWS, rook would enable that.

Post reply on HN