Live data from Hacker News

K0s – Zero Friction Kubernetes

github.com

21–30 of 78 posts

Re: K0s – Zero Friction Kubernetes

#21
post #20
post #14

Earlier quoted context omitted.

Doesn't Compose take care of that with "restart: always"? Though that probably doesn't work on reboot...

"restart: always" and "restart: unless-stopped" both work fine after reboot on my system. (Although Docker recommends managing it using systemd or upstart if you want more granular control). https://docs.docker.com/config/containers/start-containers-a...

This is for Docker, right? Not Compose? Though I guess Compose would just set the flag in Docker, in which case it'd do the same thing. I don't want more granularity than "unless stopped", basically, so this is ideal, thanks!

Re: K0s – Zero Friction Kubernetes

#22
post #6

Has anyone used K3s or this? Are they a good alternative to Dokku? I'm looking for something to either deploy on a server to host all my sideprojects (though Dokku has been basically perfect so far), or on my home server for various things like Gitea, Nextcloud and random scripts. Is K3s/K0s a good idea for either of those?

I’m a fan of Hashicorp Nomad (coupled with Vault and Consul). It’s a matter of preference though, if you love the kubernetes abstractions and APIs you’re probably better off with k3s/k0s. I personally very much prefer Nomads less obfuscating abstraction layers. This is coming from 2-3 years of managed k8s at work.

It does take a bit of effort to set up ”properly“ - ideally each of vault/consul/nomad server should run on 3 separate instance for HA quorum, for a total of 9 machines apart from the server (“client”) you’re actually running stuff on. For a casual homelab this is obviously severe overkill. You don’t need to. You can run everything perfectly fine as single instance on the same machine.

Personally I went a bit nuts with HA so I have 9 single-board-computers apart from my workload runners. You really don’t need to if OK with downtime in event of server restarts and maintenance, though.

Re: K0s – Zero Friction Kubernetes

#23
post #13
post #10

Earlier quoted context omitted.

Ah, I see, thank you. Sounds like just manually starting docker-compose stuff will be preferable right now.

I'd make systemd unit files around the docker-compose deployments. Upload everything with something like Ansible. If you want to go perfect go for NixOS instead, but that might be controversial.

I think the main question is: do you need orchestration? Ie scheduling workloads dynamically to different machines. If yes, then ks/Nomad/Swarm. If no and you want minimal overhead then the above is the answer.

I’d also suggest Podman as a Docker alternative, especially for these scenarios. Images and Dockerfiles (and many commands) are 1-to-1 compatible but there are some nice things especially with this scenario.

I wouldn’t say NixOS is controversial, at all, but it’s a very steep learning curve.

Re: K0s – Zero Friction Kubernetes

#24
post #22
post #6

Has anyone used K3s or this? Are they a good alternative to Dokku? I'm looking for something to either deploy on a server to host all my sideprojects (though Dokku has been basically perfect so far), or on my home server for various things like Gitea, Nextcloud and random scripts. Is K3s/K0s a good idea for either of those?

I’m a fan of Hashicorp Nomad (coupled with Vault and Consul). It’s a matter of preference though, if you love the kubernetes abstractions and APIs you’re probably better off with k3s/k0s. I personally very much prefer Nomads less obfuscating abstraction layers. This is coming from 2-3 years of managed k8s at work. It does take a bit of effort to set up ”properly“ - ideally each of vault/consul/nomad server should run…

This was one of the most surprising parts about Nomad for me.

I really want to like it, and for the most part I think it's a great experience, but if you're looking for a lightweight single-host PaaS-type thing, the minimum recommendation and default configuration for 3 servers is a bit wild.

I have Nomad running in single host mode on a DO box for personal stuff, and I get the distinct feeling I'm not really meant to be doing it.

3 instances minimum is a big ask IMO

Re: K0s – Zero Friction Kubernetes

#27
post #22

Earlier quoted context omitted.

I’m a fan of Hashicorp Nomad (coupled with Vault and Consul). It’s a matter of preference though, if you love the kubernetes abstractions and APIs you’re probably better off with k3s/k0s. I personally very much prefer Nomads less obfuscating abstraction layers. This is coming from 2-3 years of managed k8s at work. It does take a bit of effort to set up ”properly“ - ideally each of vault/consul/nomad server should run…

This was one of the most surprising parts about Nomad for me. I really want to like it, and for the most part I think it's a great experience, but if you're looking for a lightweight single-host PaaS-type thing, the minimum recommendation and default configuration for 3 servers is a bit wild. I have Nomad running in single host mode on a DO box for personal stuff, and I get the distinct feeling I'm not really meant t…

Go read the best practices for Vault and the "best practices minimum" are even crazier. IME it's all fine to run all these on low-resource single instance in practice, it's just the target audience for the "minimum configuration" isn't really your homelab tinkerer or even SMB/small start-up. Consider that Hashicorp's revenue streams are consulting, enterprise licenses and, since recently, hosted PaaS. If a penny-pinching user comes with replication issues because of resource contention they can just point to those docs. RedHat does the same thing for minimum recommendations for a lot of linux software that in practice will work perfectly fine on much smaller machines.

Your single host mode DO box is most likely perfectly fine. You just won't have the availability benefits during downtime of that instance but, well, duh.

I guess the one practical catch is that if you feel strongly about security and noise neighbor issues, you really should at least separate out Consul Server/Vault Server/Nomad Server into separate machines.

Just whatever you do, make sure to run an odd number of servers (1 is better than 2, 3 is better than 4), or you may run into split-brain grief. Clients are fine to scale as you see fit as they don't partake in raft consensus. So you can have a single instance "client/server", but if you add just a single new instance to schedule jobs on, make it's a pure client.

I have a dozen Nomad clients and about 20 jobs at home. My active Nomad leader is yet to use over 300 MiB RAM, 10% CPU or 0.2 load on RPi-equivalents, even during leader rotation and restarts. It's only when you start to have very high throughput of evaluations and allocations and start to put pressure on raft that you may need to increase your raft_multiplier to make it more relaxed with timeouts.

Tl;dr take the minimum configuration advice with a heap of salt.

Re: K0s – Zero Friction Kubernetes

#28
post #4
post #3

What differentiates this from k3s? Seems like there's a lot of overlap. Would be great if there was a quick summary of why someone would use this instead of k3s/microk8s/minikube/etc.

Some googling satisfied my curiosity about the k0s vs. k3s https://www.reddit.com/r/kubernetes/comments/jumaqj/k0s_yet_... https://medium.com/@saiyampathak/k0s-yet-another-kubernetes-...

Based on the medium post: Is FIPS compliance a good thing? I was under the impression that FIPS sometimes... lagged... modern ideas about security, such that if you weren't for some reason mandated to use it then it was either neutral or bad.

Re: K0s – Zero Friction Kubernetes

#29
post #27

Earlier quoted context omitted.

This was one of the most surprising parts about Nomad for me. I really want to like it, and for the most part I think it's a great experience, but if you're looking for a lightweight single-host PaaS-type thing, the minimum recommendation and default configuration for 3 servers is a bit wild. I have Nomad running in single host mode on a DO box for personal stuff, and I get the distinct feeling I'm not really meant t…

Go read the best practices for Vault and the "best practices minimum" are even crazier. IME it's all fine to run all these on low-resource single instance in practice, it's just the target audience for the "minimum configuration" isn't really your homelab tinkerer or even SMB/small start-up. Consider that Hashicorp's revenue streams are consulting, enterprise licenses and, since recently, hosted PaaS. If a penny-pinc…

For home-use, I think you could run your servers without issues (apart from that SD cards are not great at all for root or raft storage) on $20 RK3328 SBCs like a NanoPi NEO3. I may scale down to something like this at some point, given what I have now (Rpi4B-equivalents) is pretty overkill.

https://www.friendlyarm.com/index.php?route=product/product&...

Re: K0s – Zero Friction Kubernetes

#30

Is it just me or do other people also read the name as `chaos`? I guess with k8s one hard part is to get it well-configured up and running and the other to maintain it and adjust to the project's needs. But seems the project tries to improve the whole experience. Good luck (honestly)!

k8s always tripped me up, personally. I always mentally read it as "k-ubereats" (kind of like "k-means"), so it always effectively started with a "u" to me.
Post reply on HN