Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparing K3s with vanilla Kubernetes

hoelzel.it

Post reply on HN
Print view View original

41–50 of 90 posts

Re: Comparing K3s with vanilla Kubernetes

#41

Post by davidkuennen »

wouldbecouldbe wrote: post #8

It was 01:30. About to go to bed. Next day would fly with my daughter to holidays. I checked the apps of a bunch of clients. None of them loaded. I was like what... I checked the server. Everything down. I'd been running Kubernetes on Digital Ocean. And Digital Ocean forced a Kubernetes update that was incompatitable with mine at night. Took me 8 hours to fix it. No sleep. Ended up moving it back to a good old VPS. A…

Sounds like Digital Ocean was the problem, not Kubernetes.

Been running managed Kubernetes in GCP for years without any issues.

They are bugging me to update for some time now, but I don't think they would force an update on me.

Re: Comparing K3s with vanilla Kubernetes

#42

Post by zzyzxd »

cwayne wrote: post #38
zzyzxd wrote: post #31

I tried k3s several times in the past few years but I still can't use it in my homelab: 1. the project claims to be production ready and support HA control plane setup, but there's no solution for API load balancing out of box. How do you bring up a new node(either control plane or worker node)? You write down the join token produced by the first control plane node, and hardcode the token and the existing control pla…

Could check out https://github.com/k3s-io/k3s-ansible althought admittedly it's a bit in need of some lovin'

Yes, I did. Unfortunately, that playbook only supports single master.

Re: Comparing K3s with vanilla Kubernetes

#43

Post by mhink »

mulmen wrote: post #11
apetresc wrote: post #6

Earlier quoted context omitted.

That's the joke, though? It's k8s, but "smaller".

What are the three letters? It only makes sense for being confusing, which is the base state of Kubernetes. "Our name is literally a complex joke about complexity" doesn't inspire confidence in a project that supposedly simplifies complexity.

They took a chunk out of the left side of the "8", leaving "3". At least, that's how I've always interpreted it.

Re: Comparing K3s with vanilla Kubernetes

#44

Post by jhoelzel »

kamikazechaser wrote: post #39

Title is misleading. k3s is a deployment stack/distribution that builds off various Kubernetes modules. It must pass a certain test suite to conform to Kubernetes standards. What you might be trying to compare is kubeadm which is the official deployment stack provided by Kubernetes.

while somewhat yes but no.

im not trying to compare it with kubeadm (which is a more a setup script https://kubernetes.io/docs/reference/setup-tools/kubeadm/ ) but with the fact that vanilla kubernetes comes with moving parts that have to be configured and maintained and also updated separately.

you can actually setup "kubernetes" which is often referred to as vanilla kubernetes without it too. See "Kubernetes the hard way" by kelsey hightower.

Re: Comparing K3s with vanilla Kubernetes

#45

Post by ph4te »

zzyzxd wrote: post #31

I tried k3s several times in the past few years but I still can't use it in my homelab: 1. the project claims to be production ready and support HA control plane setup, but there's no solution for API load balancing out of box. How do you bring up a new node(either control plane or worker node)? You write down the join token produced by the first control plane node, and hardcode the token and the existing control pla…

I've gone through this a few times recently and have it in my homelab and at the office. What works good for me using kube-vip to a VIP on the control plane, and then MetalLB to dish out private addresses in the respective networks, or even statically assigned addresses. I have been turning them all up with k3sup which works like a charm.

turn up the first node, install kube-vip, switch config to point to the vip, turn up all my other master nodes, then turn up my workers, install metallb, setup my subnet, install rancher, expose it with a LB, install longhorn. then start deploying things. here is an example of what i use to turn up the first one with k3sup. all of the servers are turned up and configured with ansible doing minimal updates, users, sudo access, etc..

k3sup install \ --ip=192.168.1.11 \ --user=k3s-user \ --sudo \ --tls-san=192.168.1.10 \ --cluster \ --k3s-channel=stable \ --k3s-version=v1.24.12+k3s1 \ --no-extras \ --k3s-extra-args "--flannel-iface=ens160 --node-ip=192.168.1.11" \ --merge \ --local-path $HOME/.kube/config \ --context=k3s-lab

Re: Comparing K3s with vanilla Kubernetes

#46

Post by cwayne »

mulmen wrote: post #18
sakisv wrote: post #16

Earlier quoted context omitted.

Well, for me the whole replace-letters-with-numbers thing is idiotic and requires some inside knowledge to understand which letters are replaced by all the numbers in words like a11y i19n k8s, etc. On the bright side you can put whatever you want there. Personally I read k3s as "kerts", because why not. Similar to "thirds" but with the "t" from kubernetes.

I choose to read k3s as "soft boiled egg" because that makes as much sense as anything.

as the manager of the team that maintains it, this is now canon. We will use this exclusively in internal communications. thank you.

Re: Comparing K3s with vanilla Kubernetes

#47

Post by zzyzxd »

jhoelzel wrote: post #34
zzyzxd wrote: post #31

I tried k3s several times in the past few years but I still can't use it in my homelab: 1. the project claims to be production ready and support HA control plane setup, but there's no solution for API load balancing out of box. How do you bring up a new node(either control plane or worker node)? You write down the join token produced by the first control plane node, and hardcode the token and the existing control pla…

Truth be told yes that tricky but can be managed with ansible easily for instance. 1) Your main problem though would probably be the need for a haproxy or bgp which does load balancing for you. There are other solutions like kube-vip but they are more a "failover" solution that HA. Which would be fine for a homelab and is for instance how Rancher Harvester (kubernetes for virtual machines) does it. 2) you have to pas…

Yup, when I said that I was half way there, I was trying to use ansible to deploy a nginx systemd unit on every node to load-balancing requests to API servers. Many other distributions can do this with a static pod managed by local kubelet, but that's not possible in k3s.

I know there are solutions to my problem, but I can either implement them by myself, or like you mentioned, I can just use a different distribution and not having these problems.

Re: Comparing K3s with vanilla Kubernetes

#48

Post by tinco »

davidkuennen wrote: post #41
wouldbecouldbe wrote: post #8

It was 01:30. About to go to bed. Next day would fly with my daughter to holidays. I checked the apps of a bunch of clients. None of them loaded. I was like what... I checked the server. Everything down. I'd been running Kubernetes on Digital Ocean. And Digital Ocean forced a Kubernetes update that was incompatitable with mine at night. Took me 8 hours to fix it. No sleep. Ended up moving it back to a good old VPS. A…

Sounds like Digital Ocean was the problem, not Kubernetes. Been running managed Kubernetes in GCP for years without any issues. They are bugging me to update for some time now, but I don't think they would force an update on me.

I hope you're right, I haven't had time to look at the same warning for 6 months now. There was a hard deadline on it in February and it came and went. The backwards incompatible Kubernetes api updates are a PITA. I know they said "beta", but if you literally have a version field in your manifest format, why ever deprecate a way of configuring something? They could just have operators translate the configs automatically on the fly.

Re: Comparing K3s with vanilla Kubernetes

#49

Post by infogulch »

mulmen wrote: post #11
apetresc wrote: post #6

Earlier quoted context omitted.

That's the joke, though? It's k8s, but "smaller".

What are the three letters? It only makes sense for being confusing, which is the base state of Kubernetes. "Our name is literally a complex joke about complexity" doesn't inspire confidence in a project that supposedly simplifies complexity.

kubernetes -> k8s

k8s -> k3s

Double numeronym'd

Re: Comparing K3s with vanilla Kubernetes

#50

Post by alexellisuk »

I'm a big fan of K3s, however managed Kubernetes from a large cloud vendor with a track record has a lot to offer when it comes to reducing management and the need for an SRE for K8s itself.

Folks might also be interested in two free resources:

1 - K3sup https://github.com/alexellis/k3sup - the author mentions HA K3s - K3sup is an easy way to get that using SSH. It's also a good pairing for K3s with Raspberry Pi 2 - Kubernetes at the Edge with K3s (CNCF / LF course) - I was commissioned to write this and I talk a lot about the differences and also the origin story of K3s and what Darren was aiming for.

Have fun with Kubernetes - whichever flavour you go for.

Post reply on HN