Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

jeffgeerling.com

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#71

Post by yipbub »

ownagefool wrote: post #37
bogomipz wrote: post #34

Earlier quoted context omitted.

Does Rook give you the equivalent of EBS root volumes for your nodes then? Is that the function you have it providing? Does it offer something beyond using local host storage and minio? I ask because I've generally been confused about the use case for Rook despite having read the "what is Rook?" paragraph many times on the project home page. My assumption is that it lets you build your own internal cloud provider. Is…

https://rook.io/docs/rook/v1.1/ Rook will give you: - Ceph: Block Storage, Shared storage, Object storage - EdgeFS: Object but apparently will function as Block and File - Minio: Object - NFS4: Shared - Cockroach: Database - Cassandra: Database - YugabyteDB: distributed SQL database (new to me) Only the first two are marked as stable.

I'll mention that they're all independent projects under rook's umbrella.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#72

Post by bogomipz »

moondev wrote: post #44
ownagefool wrote: post #40

Earlier quoted context omitted.

I would say they're functionality quite equivalent, you just wouldn't pull them in as a root volume. You should be able to point your default StorageClass at Ceph and have it create RBD block storage devices for you, which would auto create pvc, that you mount you actual data in in your kube manifests.

The parent comment was asking about root volumes for kubernetes nodes > Does Rook give you the equivalent of EBS root volumes for your nodes then? I think we are saying the same thing if i'm not mistaken?

I only said "root volumes" as it's a common use case for EBS volumes. For instance with etcd running in a container you would want the host volume to be an EBS volume since it's critical it's a critical K8S component.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#73

Post by ownagefool »

bogomipz wrote: post #72
moondev wrote: post #44

Earlier quoted context omitted.

The parent comment was asking about root volumes for kubernetes nodes > Does Rook give you the equivalent of EBS root volumes for your nodes then? I think we are saying the same thing if i'm not mistaken?

I only said "root volumes" as it's a common use case for EBS volumes. For instance with etcd running in a container you would want the host volume to be an EBS volume since it's critical it's a critical K8S component.

Right, but you would only need to mount the etcd data partition (/var/lib/etcd afaik), rather than the entire node and/or container.

The main problem you have here, is chicken and egg. How do you use a StorageClass, kubernetes PVC, or rook to provision Block Storage for etcd, when you need etcd for kubernetes to function, and you need kubernetes to function for rook, et all.

At some point, you need to bootstrap the world, which is people either start off with cloud APIs, ansible, or PXE.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#74

Post by MuffinFlavored »

Nullabillity wrote: post #66
MuffinFlavored wrote: post #63

Earlier quoted context omitted.

I don’t fully understand where the line goes from “all I need is a Docker container for nginx + Postgres + Redis + my services” to “I need Kubernetes” When does one need to go from just Docker containers to container orchestration like k8s?

Do you care about availability? Do you care about rolling upgrades? Do you want to practice any of those two? Then it might be time to make the switch. Yes, Docker has Swarm Mode. And yes, it is easier to get started with than K8s. The problem is that it locks you into a lot of choices when you first spin up each (Docker) Service, that can't be changed without taking down the Service and then recreating it with the n…

I do blue-green deploys with Docker and I deploy the services as “app-1”, “app-2”, “app-3”, etc.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#75

Post by Nullabillity »

MuffinFlavored wrote: post #74
Nullabillity wrote: post #66

Earlier quoted context omitted.

Do you care about availability? Do you care about rolling upgrades? Do you want to practice any of those two? Then it might be time to make the switch. Yes, Docker has Swarm Mode. And yes, it is easier to get started with than K8s. The problem is that it locks you into a lot of choices when you first spin up each (Docker) Service, that can't be changed without taking down the Service and then recreating it with the n…

I do blue-green deploys with Docker and I deploy the services as “app-1”, “app-2”, “app-3”, etc.

How do you route dependencies to the correct service? If you're using a stack then you end up needing to blue-green the whole thing, if you're not then you still need to blue-green the whole dependent subset of the dependency graph.

And for anything exposed outside of the swarm you'd need to use some reverse proxy, since only one service (in the whole swarm) can bind a port at any one time.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#76

Post by fizixer »

geerlingguy wrote: post #14
raesene9 wrote: post #6

A good method for learning how clusters work and playing with them without having to spend a lot of time re-building when you break things is kind ( https://kind.sigs.k8s.io/ ). Each node is a Docker container, but the version of Kubernetes running inside it is vanilla Kubeadm, so it's quite representative of what "real" clusters would look like. The great thing about it is you can spin up a cluster in < 2 mins local…

I use Kind (and Minikube, and a number of other solutions too), but this is kind of my "Kubernetes-the-really-hard-way" fun project. Note that I maintain a parallel configuration that runs a multi-node cluster on Vagrant for local development [1] as well as a docker environment built for CI purposes[2] using almost all the same images, with the same Ansible playbooks to configure them all across platforms. [1] https:…

Sorry, late to the party.

Thanks for sharing. I fully agree with the fun, and the really-hard-way part.

However, if we ignore that part for the moment, would you think having an R-Pi cluster handy is worth it for training/sandboxing/testing with the aim of applying the skills in your day job?

Are there any advantages to having access to your own R-Pi kubernetes cluster, despite having stuff like Kind and Minikube available online?

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#78

Post by bogomipz »

ownagefool wrote: post #73
bogomipz wrote: post #72

Earlier quoted context omitted.

I only said "root volumes" as it's a common use case for EBS volumes. For instance with etcd running in a container you would want the host volume to be an EBS volume since it's critical it's a critical K8S component.

Right, but you would only need to mount the etcd data partition (/var/lib/etcd afaik), rather than the entire node and/or container. The main problem you have here, is chicken and egg. How do you use a StorageClass, kubernetes PVC, or rook to provision Block Storage for etcd, when you need etcd for kubernetes to function, and you need kubernetes to function for rook, et all. At some point, you need to bootstrap the w…

."The main problem you have here, is chicken and egg. How do you use a StorageClass, kubernetes PVC, or rook to provision Block Storage for etcd, when you need etcd for kubernetes to function, and you need kubernetes to function for rook, et all."

I totally agree. The EBS lifecycle management is generally handled by something like Terraform. That's why I was wondering if the use case for Rook is primarily bare-metal Kubernetes since AWS/GCP et al. already provide these. So I'm wondering that even in a bare-metal environment where you still need to use config management tools like Ansible/Terraform to do things like provision block storage what's the upside of Rook over existing iscsi/Ceph/minio installations?

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#79

Post by MuffinFlavored »

Nullabillity wrote: post #75
MuffinFlavored wrote: post #74

Earlier quoted context omitted.

I do blue-green deploys with Docker and I deploy the services as “app-1”, “app-2”, “app-3”, etc.

How do you route dependencies to the correct service? If you're using a stack then you end up needing to blue-green the whole thing, if you're not then you still need to blue-green the whole dependent subset of the dependency graph. And for anything exposed outside of the swarm you'd need to use some reverse proxy, since only one service (in the whole swarm) can bind a port at any one time.

> How do you route dependencies to the correct service?

Reload the nginx config with proxy_pass switch from blue -> green or green -> blue.

Re: Everything I know about Kubernetes I learned from a cluster of Raspberry Pis

#80

Post by MuffinFlavored »

imtringued wrote: post #70
MuffinFlavored wrote: post #63

Earlier quoted context omitted.

I don’t fully understand where the line goes from “all I need is a Docker container for nginx + Postgres + Redis + my services” to “I need Kubernetes” When does one need to go from just Docker containers to container orchestration like k8s?

As soon as you want load balancing for high availability but I personally am not a fan of Kubernetes.

What's an alternative?
Post reply on HN