Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Canonical introduces high-availability Micro-Kubernetes

zdnet.com

Post reply on HN
Print view View original

61–70 of 118 posts

Re: Canonical introduces high-availability Micro-Kubernetes

#61

Post by manigandham »

krimeo wrote: post #55
manigandham wrote: post #49

Earlier quoted context omitted.

How is that related? Environments and languages change. That's entirely different to running programs with zero-downtime deployments, load-balancing and traffic management, health monitoring, logging and observability, secret and config management, storage volumes, security roles, and much more. What is your replacement for all that?

> How is that related? I thought we were talking about running applications reliably. Why is complete linux userspace bundled separately in each container? > Environments and languages change. Yes. > That's entirely different to running programs with zero-downtime deployments, load-balancing and traffic management, health monitoring, logging and observability, secret and config management, storage volumes, security r…

It's a container and can be as thin or fat as you want with it's contents. You don't need to include linux inside if you don't want to. I've built containers with nothing more than a few native executables. It's just a packaging format, but easier to build and deploy than other formats like tarballs.

If you don't need K8S then don't use it. What's the problem? Run your app on your server and ignore everything else.

But most of these features have nothing to do with scale and are more about usability, reliability and consistency. Sure you can do it yourself but that's less efficient than just letting K8S do it all in one standardized way and interface.

> "I want my application to reliably and quickly serve my customers and be easy to maintain and debug."

That's what K8S helps with. I've spent 10 years running large distributed applications handling billions of requests per day in multiple regions. I don't care about the ABI and don't see why that's relevant, but I do know that K8S has made many things easier in actually running these apps.

Re: Canonical introduces high-availability Micro-Kubernetes

#62

Post by peterwwillis »

jrockway wrote: post #7
candiddevmike wrote: post #6

This is very interesting, we're seeing a lot of Kubernetes "flavors" coming out that remove the etcd requirement. It's no secret that etcd is a key part of why Kubernetes is complex--etcd scaling/securing/recovery is really hard. I think Kubernetes would do well to make swapping out the storage backend possible without having all of these forks. Kubernetes is too tightly coupled to etcd, and for little benefit. I wou…

To some extent, the distributed consensus is kind of the important part. If you have a bunch of components that don't know what state they're supposed to be in, you don't really have a cluster. Looking at the marketing documentation (I didn't read the code), they just wrote their own consensus and datastore instead of using etcd. So the underlying fundamental computer science problems still exist, but nobody has been…

You don't need distributed consensus to have a cluster of reliable services. We've had large-scale clusters of applications for what, 30 years? And distributed consensus being generally used by a few players for about 10 of those. Etcd is neat, but in no way mandatory for what it was even invented for. 99% of people just don't need it.

Re: Canonical introduces high-availability Micro-Kubernetes

#63

Post by seabrookmx »

bryanlarsen wrote: post #60
pmahoney wrote: post #12

Earlier quoted context omitted.

This doesn't answer your question, but I'm piggy-backing with comments on k3s. For me, using k3s for development (not prod), the killer feature is running it in --docker mode where the node uses the local dockerd to run containers (vs. managing its own containerd instance). This allows building images locally with `docker build` and immediately using them in kubernetes pods _without_ first pushing to a (possibly loca…

A couple of years ago minikube supported this with --vm-driver=none

Still does: https://minikube.sigs.k8s.io/docs/drivers/

Re: Canonical introduces high-availability Micro-Kubernetes

#64

Post by moondev »

thor24 wrote: post #18
koeng wrote: post #4

Earlier quoted context omitted.

I think minikube is for local usage. I use K3S for deploying an actual production cluster, so I’m more curious about how micro-k8s acts on that scale

> I use K3S for deploying an actual production cluster You mean when you want to run a small cluster of let's say less than 10 nodes (anything in single digits)? Why doesn't normal k8 work this way? like same tech but just less scale? (I should probably read more on the side myself as well, new to this K8 world).

It does. Upstream kubeadm - authentic kubernetes - can even run on a single node. Not sure why people choose k3s or microk8s when you can just as easily deploy the real thing.

Re: Canonical introduces high-availability Micro-Kubernetes

#66

Post by jrockway »

peterwwillis wrote: post #62
jrockway wrote: post #7

Earlier quoted context omitted.

To some extent, the distributed consensus is kind of the important part. If you have a bunch of components that don't know what state they're supposed to be in, you don't really have a cluster. Looking at the marketing documentation (I didn't read the code), they just wrote their own consensus and datastore instead of using etcd. So the underlying fundamental computer science problems still exist, but nobody has been…

You don't need distributed consensus to have a cluster of reliable services. We've had large-scale clusters of applications for what, 30 years? And distributed consensus being generally used by a few players for about 10 of those. Etcd is neat, but in no way mandatory for what it was even invented for. 99% of people just don't need it.

I don't know of any clustering system that works without some sort of leader. In many cases, the leader is you. If your computer dies, you can walk to another one and still deploy software. If you disappear, that's a Big Problem, however.

No modern clustering system is much different. Services aren't killed off if a leader can't be elected. It's just that having all your servers alive but uncontrollable is almost as scary as them being down.

Re: Canonical introduces high-availability Micro-Kubernetes

#67

Post by tuananh »

gavinray wrote: post #35

I am all for this trend. Microk8s and k3s are both a joy to develop with (though I have ran into a few bugs with k3s so tend to prefer Microk8s). How many folks are running self-managed k8s in production though, out of curiosity? It seems so economical to deploy k3s or use something like Rancher on dirt cheap VPS's from some place like Hetzner -- but what's the ops burden and failure risk like? Never tried it myself…

> (though I have ran into a few bugs with k3s so tend to prefer Microk8s).

i tried both in my homelab and this is my experience as well. microk8s seems to has less bug for me.

Re: Canonical introduces high-availability Micro-Kubernetes

#68

Post by wahern »

jrockway wrote: post #66
peterwwillis wrote: post #62

Earlier quoted context omitted.

You don't need distributed consensus to have a cluster of reliable services. We've had large-scale clusters of applications for what, 30 years? And distributed consensus being generally used by a few players for about 10 of those. Etcd is neat, but in no way mandatory for what it was even invented for. 99% of people just don't need it.

I don't know of any clustering system that works without some sort of leader. In many cases, the leader is you. If your computer dies, you can walk to another one and still deploy software. If you disappear, that's a Big Problem, however. No modern clustering system is much different. Services aren't killed off if a leader can't be elected. It's just that having all your servers alive but uncontrollable is almost as…

Distributed consensus doesn't require a leader. The same process used to elect a leader (i.e. change leader state) can simply be used to change state generally. See, e.g., There Is More Consensus in Egalitarian Parliaments: https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf

In fact, for something like what etcd is used for in k8s, as the root and lynchpin of all state, but not direct application I/O (pods use their own data stores, and even for cluster management etcd usually just contains pointers to external data), a leaderless architecture makes the most sense. Protocols to choose a leader are latency and throughput optimizations, but the core state for a cluster (members, pod metadata, etc) shouldn't require very much state and therefore should be relatively low traffic as compared to most database applications. And as shown in the above paper, for certain scenarios (including, arguably, the k8s scenario) a leaderless architecture can have better latency and throughput, not to mention availability.

Re: Canonical introduces high-availability Micro-Kubernetes

#69

Post by SomaticPirate »

candiddevmike wrote: post #6

This is very interesting, we're seeing a lot of Kubernetes "flavors" coming out that remove the etcd requirement. It's no secret that etcd is a key part of why Kubernetes is complex--etcd scaling/securing/recovery is really hard. I think Kubernetes would do well to make swapping out the storage backend possible without having all of these forks. Kubernetes is too tightly coupled to etcd, and for little benefit. I wou…

https://github.com/rancher/kine provides a translation layer for etcd that supports various SQL flavors. It is currently used only by k3s afaik

Re: Canonical introduces high-availability Micro-Kubernetes

#70

Post by wahern »

candiddevmike wrote: post #15
jrockway wrote: post #7

Earlier quoted context omitted.

To some extent, the distributed consensus is kind of the important part. If you have a bunch of components that don't know what state they're supposed to be in, you don't really have a cluster. Looking at the marketing documentation (I didn't read the code), they just wrote their own consensus and datastore instead of using etcd. So the underlying fundamental computer science problems still exist, but nobody has been…

If etcd blows up, your cluster should stay in the same state it was in before etcd blew up.

Presuming, of course, that your cluster is completely isolated from the outside world. Otherwise, the interaction of various controllers, deployment stacks, and application usage conspire to generate endless amounts of cluster state change requests, which can quickly pileup if etcd is unavailable or behaving erratically and cause all manner of havoc. Containers can fail to restart after crashing; jobs won't run, etc. Plus, in k8s etcd is typically used for more than just the barebones state required for basic node and pod status. Metrics, API requests, DNS, etc, often depend on etcd--the same etcd--for storage and communication, which can compound the effect of a flakey etcd and even instigate it.
Post reply on HN