Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

K0s – Zero Friction Kubernetes

github.com

Post reply on HN
Print view View original

71–78 of 78 posts

Re: K0s – Zero Friction Kubernetes

#71

Post by oneplane »

ralph87 wrote: post #34
oneplane wrote: post #12

I don't understand why people seem to want K8s but then find it 'difficult'. It's really not that complex, and when you add stuff on to it, that is your own choice (just like with k3s and k0s). You essentially end up with a kubelet, apiserver (with etcd) and two or three copy-and-paste YAML files (manifests) to setup a 'full' k8s cluster. If that is too much, is k8s even relevant to your case? Say your workload is so…

Kubernetes does a whole lot more than just restart jobs when a node fails, in fact of all its features this is one most people probably see in action the least. Kubernetes centralizes all the traditional technical nonsense related to providing a robust environment to deploy applications to. I want Kubernetes even in a single node scenario because I want Kubernetes-like packaging, deployment and network services for a…

Again, what is it for? If you have a single server, what is K8S going to give you over say, a single static binary or a docker container?

K8S on a single node does nothing for you network-wise, you have no overlay network (because there is nothing to overlay) you have no ingress or egress (there is only 1 node so no matter what you 'configure' your ingress and egress will be that same node), and it's unlikely to have enough resources to do something like a full application deployment with some big helm chart.

While I agree that you can (ab)use K8S as a runtime and packaging format, all of those big benefits are removed when you are running it on 1 node, except perhaps the fact that you can talk to the apiserver and define your jobs/tasks/pods the same way. But even then you'd only do that locally, because 'testing' in a dev or staging env that doesn't match prod is going to give you non-representative results.

Re: K0s – Zero Friction Kubernetes

#72

Post by ralph87 »

oneplane wrote: post #71
ralph87 wrote: post #34

Earlier quoted context omitted.

Kubernetes does a whole lot more than just restart jobs when a node fails, in fact of all its features this is one most people probably see in action the least. Kubernetes centralizes all the traditional technical nonsense related to providing a robust environment to deploy applications to. I want Kubernetes even in a single node scenario because I want Kubernetes-like packaging, deployment and network services for a…

Again, what is it for? If you have a single server, what is K8S going to give you over say, a single static binary or a docker container? K8S on a single node does nothing for you network-wise, you have no overlay network (because there is nothing to overlay) you have no ingress or egress (there is only 1 node so no matter what you 'configure' your ingress and egress will be that same node), and it's unlikely to have…

Ever tried to host multiple apps on a single machine? Oh look, a custom Nginx config only one person understands. Oh look, some hacked up letsencrypt config only one person understands, etc etc.

> K8S on a single node does nothing for you network-wise

- Container IP auto-assignment

- Container security policy

- Container DNS management

- Ingress management ("custom Nginx config")

- "Environment that feels like a large network and doesn't change if moved to a large network"

What part of this is difficult to understand?

Re: K0s – Zero Friction Kubernetes

#73

Post by oneplane »

ralph87 wrote: post #72
oneplane wrote: post #71

Earlier quoted context omitted.

Again, what is it for? If you have a single server, what is K8S going to give you over say, a single static binary or a docker container? K8S on a single node does nothing for you network-wise, you have no overlay network (because there is nothing to overlay) you have no ingress or egress (there is only 1 node so no matter what you 'configure' your ingress and egress will be that same node), and it's unlikely to have…

Ever tried to host multiple apps on a single machine? Oh look, a custom Nginx config only one person understands. Oh look, some hacked up letsencrypt config only one person understands, etc etc. > K8S on a single node does nothing for you network-wise - Container IP auto-assignment - Container security policy - Container DNS management - Ingress management ("custom Nginx config") - "Environment that feels like a larg…

> Ever tried to host multiple apps on a single machine? Yep works fine. Has been for decades, even before containers existed. Guess what the sites-available and sites-enabled directories for apache and nginx are for.

> Oh look, a custom Nginx config only one person understands.

Just because you put it in a container doesn't mean it's no longer custom or that everyone suddenly understands it.

> Oh look, some hacked up letsencrypt config only one person understands, etc etc.

Plenty of people put their nasty hacks in containers and pod definitions and still nobody (or just 1 person) understands it. Packaging changes none of this; a dirty pod, container, VM image it's all still dirty.

> K8S on a single node does nothing for you network-wise > - Container IP auto-assignment

So does docker, or even an uncontainerized bridge interface

> - Container security policy

So does Docker, or a plain cgroup

> - Container DNS management Yep, that it does. But when you only have 1 node, what is the point?

> - Ingress management ("custom Nginx config")

Great, but besides moving complexity from your app to the infra it doesn't help at all on a single node. It actually gets worse: node goes down, everything goes down (app, fallback, load balancing, routing, security)

> - "Environment that feels like a large network and doesn't change if moved to a large network"

So unless you are doing some local development that you later on push to dev/prod, we're talking about feelings. Not much objective to say about that except that it exists.

> What part of this is difficult to understand?

All of it. Shoving complexity and responsibility around doesn't reduce it, and having people make bad software isn't less bad because of the runtime it runs on.

Kubernetes in prod is great, and the envs that go with it (like development and staging), sure. But when you run something in prod, and you need availability, scalability and a host of standardised facilities, then a single node or some magic 'it works by default' config is very far removed from real-world production.

Re: K0s – Zero Friction Kubernetes

#74

Post by ralph87 »

oneplane wrote: post #73
ralph87 wrote: post #72

Earlier quoted context omitted.

Ever tried to host multiple apps on a single machine? Oh look, a custom Nginx config only one person understands. Oh look, some hacked up letsencrypt config only one person understands, etc etc. > K8S on a single node does nothing for you network-wise - Container IP auto-assignment - Container security policy - Container DNS management - Ingress management ("custom Nginx config") - "Environment that feels like a larg…

> Ever tried to host multiple apps on a single machine? Yep works fine. Has been for decades, even before containers existed. Guess what the sites-available and sites-enabled directories for apache and nginx are for. > Oh look, a custom Nginx config only one person understands. Just because you put it in a container doesn't mean it's no longer custom or that everyone suddenly understands it. > Oh look, some hacked up…

We could argue this ad infinitum, K8s of course doesn't remove all proprietary elements from a solution but it is a huge step up. Speaking as an ex-Googler, it took over 10 years but I'm so happy the rest of the world finally has a standard like this, the world is a better place for it even though I at one point had to unlearn all my traditional sysadmin habits and immerse myself in an environment practising it successfully to finally understand.

Your original question was what is the point. These are the points. As for why not Docker, k8s network effects and strategy of its sponsors mean Docker is on a lifeline, everyone knows that.

Re: K0s – Zero Friction Kubernetes

#75

Post by josegonzalez »

SheinhardtWigCo wrote: post #17
pas wrote: post #8

Earlier quoted context omitted.

I have tried k3s and microk8s. Recently setup ArgoCD, connected to our GitLab instance. Tried both Traefik and Nginx Ingress controllers. And so on. And dokku is better if you want a PaaS. There are probably dokku/heroku/deis-like things that run on kubernetes, but so far I haven't found 'it'. Of course you can just run dokku on any k8s distribution. You can even use k8s as a backend for dokku instead of docker - htt…

dokku is fun to play with but I found it too buggy for use in production.

Would you mind pointing out a few things that were buggy about your experience with Dokku?

Re: K0s – Zero Friction Kubernetes

#76

Post by SheinhardtWigCo »

josegonzalez wrote: post #75
SheinhardtWigCo wrote: post #17

Earlier quoted context omitted.

dokku is fun to play with but I found it too buggy for use in production.

Would you mind pointing out a few things that were buggy about your experience with Dokku?

Deploys occasionally failing and leaving behind a lock file, requiring manual intervention.

Let’s Encrypt integration not working properly without dokku-event-listener, which must be built & installed from source and is basically undocumented.

At the time I used it, blue/green deployments were not available. (I see this is now supported, great to see that.)

To be fair, I didn’t report any of these issues, so I may have done something wrong or there may have been solutions that I missed. This was also 6+ months ago.

I also think that with all the major cloud providers now supporting Kubernetes as a first-class primitive, and with there being an abundance of CI/CD tools that can trivially deploy to Kubernetes, the PaaS concept is no longer as attractive as it once was.

Re: K0s – Zero Friction Kubernetes

#77

Post by oneplane »

ralph87 wrote: post #74
oneplane wrote: post #73

Earlier quoted context omitted.

> Ever tried to host multiple apps on a single machine? Yep works fine. Has been for decades, even before containers existed. Guess what the sites-available and sites-enabled directories for apache and nginx are for. > Oh look, a custom Nginx config only one person understands. Just because you put it in a container doesn't mean it's no longer custom or that everyone suddenly understands it. > Oh look, some hacked up…

We could argue this ad infinitum, K8s of course doesn't remove all proprietary elements from a solution but it is a huge step up. Speaking as an ex-Googler, it took over 10 years but I'm so happy the rest of the world finally has a standard like this, the world is a better place for it even though I at one point had to unlearn all my traditional sysadmin habits and immerse myself in an environment practising it succe…

Of course Docker is doomed ;-) We have CRI-O, gvisor etc. showing that it works fine without it as well. Someone implemented a OCI compatible image runner in bash using standard cgroups, with a bit of luck we'll end up calling containers 'containers' and images 'images' instead of using docker's brand name.

Also, I'm not saying that k8s is bad, or using k8s as a practical API definition of the platform to target when packaging and configuring applications; I was aiming at the 'boo hoo k8s is too hard' tagline every "simple" version seems to hold on to.

One could also install standard K8S and remove the taint and run pods locally, same result.

Re: K0s – Zero Friction Kubernetes

#78

Post by josegonzalez »

SheinhardtWigCo wrote: post #76
josegonzalez wrote: post #75

Earlier quoted context omitted.

Would you mind pointing out a few things that were buggy about your experience with Dokku?

Deploys occasionally failing and leaving behind a lock file, requiring manual intervention. Let’s Encrypt integration not working properly without dokku-event-listener, which must be built & installed from source and is basically undocumented. At the time I used it, blue/green deployments were not available. (I see this is now supported, great to see that.) To be fair, I didn’t report any of these issues, so I may ha…

> Deploys occasionally failing and leaving behind a lock file, requiring manual intervention.

Interesting, would have been good to have these reported. Aside, unlocking an app should just be an `apps:unlock` call.

> Let’s Encrypt integration not working properly without dokku-event-listener, which must be built & installed from source and is basically undocumented.

That shouldn't be the case - the letsencrypt plugin was available long before the dokku-event-listener project was, and the latter has been built/released automatically since April[1] (though didn't have documentation on it's internals till recently, as that wasn't really necessary for end users as there isn't any configuration).

> At the time I used it, blue/green deployments were not available. (I see this is now supported, great to see that.)

Not sure what youre referencing here. We don't have anything specific for blue/green deployments, but we have had zero-downtime checks for a while. If you can point out to me where you found this (or what you were expecting) that would be awesome :)

> To be fair, I didn’t report any of these issues, so I may have done something wrong or there may have been solutions that I missed. This was also 6+ months ago.

Reporting issues is always appreciated as its almost certain there were bugs on our end, but it seems you've found your solution :)

> I also think that with all the major cloud providers now supporting Kubernetes as a first-class primitive, and with there being an abundance of CI/CD tools that can trivially deploy to Kubernetes, the PaaS concept is no longer as attractive as it once was.

I don't think this is quite true. Kubernetes is still a large system that doesn't come "batteries included", and every setup is at least subtly different from the next. I think a large number of teams spend a disproportionate amount of time configuring their preferred toolchain - and then maintaining it - once you get to a certain scale. Additionally, a kubernetes manifest is much lower-level than is necessary for the general app developer who just wants to get things done.

I'm personally hoping that Kubernetes distributions begin to mature more (Tanzu and Openshift are examples, but not the only ones) so that folks start worrying less about operating and maintaining clusters and more on either the underlying infra or the end developer experience.

Post reply on HN