Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

K3k: Kubernetes in Kubernetes

github.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: K3k: Kubernetes in Kubernetes

#33

Post by mystifyingpoi »

2ndorderthought wrote: post #31

Can someone explain what this even means? Explain it like I am a software engineer with 20 years experience who has not yet found a strong use case for running kubernetes outside of hand holding cloud provider options

This is extremely niche. 99.9% of Kubernetes deployments will never need such nesting. It could be useful for testing tooling (I guess maybe operators?) without recreating the "top-level" cluster all the time.

Also it's a fun idea. Sandbox in a sandbox.

Re: K3k: Kubernetes in Kubernetes

#35

Post by phrotoma »

2ndorderthought wrote: post #31

Can someone explain what this even means? Explain it like I am a software engineer with 20 years experience who has not yet found a strong use case for running kubernetes outside of hand holding cloud provider options

K8s encourages thinking about workloads as "cattle not pets". App running in K8s falls over? Blow it away and let K8s recreate it, etc.

However clusers themselves often become the new pets. Many orgs do not reach a level of operational maturity where they can blow away and recreate whole clusters without downtime and toil.

A meta-pattern has emerged where higher order tooling managers a whole fleet of clusters. This is an implementation of that meta pattern which uses K8s itself as the higher order tool to manage other clusters.

It's not a new idea, just a new implementation of the pattern.

Re: K3k: Kubernetes in Kubernetes

#36

Post by lateral_cloud »

randomtoast wrote: post #19

This type of approach carries a significantly higher operational risk compared to operating multiple Kubernetes clusters on separate VMs or physical hardware. If you eventually update the main Kubernetes cluster that manages the virtual clusters and something goes wrong, you could potentially bring down your entire fleet of Kubernetes clusters all at once.

I don't think this is intended for production

Re: K3k: Kubernetes in Kubernetes

#37

Post by 2ndorderthought »

phrotoma wrote: post #35
2ndorderthought wrote: post #31

Can someone explain what this even means? Explain it like I am a software engineer with 20 years experience who has not yet found a strong use case for running kubernetes outside of hand holding cloud provider options

K8s encourages thinking about workloads as "cattle not pets". App running in K8s falls over? Blow it away and let K8s recreate it, etc. However clusers themselves often become the new pets. Many orgs do not reach a level of operational maturity where they can blow away and recreate whole clusters without downtime and toil. A meta-pattern has emerged where higher order tooling managers a whole fleet of clusters. This…

Thank you. Wow I had no idea this was a problem. Seems kind of nightmare territory. In a weird way it makes me respect elixir/erlang even more. It's not the exact same problem obviously but really had me thinking about beam etc

Re: K3k: Kubernetes in Kubernetes

#38

Post by weitzj »

enrichman wrote: post #28
weitzj wrote: post #24

Earlier quoted context omitted.

Thank you for your feedback. I understood from the host cluster perspective you won’t see the child cluster pods. And what is the perspective on nodes? Can you have like a host cluster spawning on host nodes and the host cluster has control over spawning separate physical nodes which contain the child cluster (api server) + workload pods ?

As I understand it, the virtual cluster pods are treated as standard workloads by the host. This means if you scale the nodes up or down, they will be rescheduled accordingly. You can currently use node selectors to manage this behavior, though we are developing a more flexible approach using affinity rules.

Thank you

Re: K3k: Kubernetes in Kubernetes

#39

Post by dboreham »

2ndorderthought wrote: post #37
phrotoma wrote: post #35

Earlier quoted context omitted.

K8s encourages thinking about workloads as "cattle not pets". App running in K8s falls over? Blow it away and let K8s recreate it, etc. However clusers themselves often become the new pets. Many orgs do not reach a level of operational maturity where they can blow away and recreate whole clusters without downtime and toil. A meta-pattern has emerged where higher order tooling managers a whole fleet of clusters. This…

Thank you. Wow I had no idea this was a problem. Seems kind of nightmare territory. In a weird way it makes me respect elixir/erlang even more. It's not the exact same problem obviously but really had me thinking about beam etc

Imagine you are the developer of k8s hosted systems. Now imagine you want to test your systems in a repeatable fashion. You'd need some way to spin up a test k8s cluster, deploy your application, subject it to a test workload. That's simple and easy if you only need one physical cluster node: you can use k3s or perhaps kind. But if you want multiple physical nodes, not so easy. This solves that problem by leveraging an existing k8s cluster, which is a standard thing easily obtained. You might now ask why not just use that cluster (why the terduckin?) Answer: cost, time, hassle, you want a different version of k8s than the hosting provider gives you.
Post reply on HN