I can bootstrap an entire RKE2 VM (VM + RKE2 + join cluster) in like 5 mins with Salt (although I have no reason to think you couldn't do it with Ansible).
It's a cool project, but I didn't think the K3s part was the hard part.
K3sup – bootstrap K3s over SSH in < 60s
11–20 of 55 posts
Re: K3sup – bootstrap K3s over SSH in < 60s
#12Earlier quoted context omitted.
Do you have a writeup what problems you ran into?
We do, let me check with my team and post it here. There were many issues. On top of my mind was, after a DR drill where in a VM was booted, node did not join the cluster. Apart from that bunch of issues due to etcd, longhorn. Another major one was the CNI stopped work for a particular node. Garbage collection for images was another, we labelled the images, it would still remove then from the node. Bunch of these kin…
Meanwhile our architecture team that surely supported 0 real life k8s went with no vendor, on premises deployments, claiming it was as easy as booting a VM, after 2y, there is 2 apps running and supposedly all future apps will be deployed on that cluster.
I cannot wait for the end of this month to leave that place.
Re: K3sup – bootstrap K3s over SSH in < 60s
#13Earlier quoted context omitted.
We do, let me check with my team and post it here. There were many issues. On top of my mind was, after a DR drill where in a VM was booted, node did not join the cluster. Apart from that bunch of issues due to etcd, longhorn. Another major one was the CNI stopped work for a particular node. Garbage collection for images was another, we labelled the images, it would still remove then from the node. Bunch of these kin…
Meanwhile our architecture team that surely supported 0 real life k8s went with no vendor, on premises deployments, claiming it was as easy as booting a VM, after 2y, there is 2 apps running and supposedly all future apps will be deployed on that cluster. I cannot wait for the end of this month to leave that place.
Anything real world tech is hard.
We are hiring, btw.
Re: K3sup – bootstrap K3s over SSH in < 60s
#14I used this for a bit a few years ago but eventually needed something that was hard or impossible in k3sup and just went to using the k3s tools directly. My deployment script actually got simpler after removing k3sup. Also, fun fact, k3sup is pronounced "ketchup" according to the README[0] [0]: https://github.com/alexellis/k3sup/blob/master/README.md
The pronunciation ketchup is somewhat unfortunate as a popular backup operator for k8s, k8up, also claims this.
Re: K3sup – bootstrap K3s over SSH in < 60s
#15[flagged]
Re: K3sup – bootstrap K3s over SSH in < 60s
#16I'm trying to understand why people are spinning up so many k8s clusters that they need a tool to do it for them?
I have one. And it's managed. I don't think there's significant cost savings to going unmanaged, but maybe. Even so, why would I need a ton of them?
Re: K3sup – bootstrap K3s over SSH in < 60s
#17I'm trying to understand why people are spinning up so many k8s clusters that they need a tool to do it for them? I have one. And it's managed. I don't think there's significant cost savings to going unmanaged, but maybe. Even so, why would I need a ton of them?
You're cool if you manage your own K8S cluster.
Re: K3sup – bootstrap K3s over SSH in < 60s
#18I'm trying to understand why people are spinning up so many k8s clusters that they need a tool to do it for them? I have one. And it's managed. I don't think there's significant cost savings to going unmanaged, but maybe. Even so, why would I need a ton of them?
It's applied big brain memetics. k8s turned pet servers into cattle. People then do the next step and want to treat their clusters as cattle as well. Also it has a bit of the "can it run DOOM" vibe to treat whole k8s clusters like this.
Re: K3sup – bootstrap K3s over SSH in < 60s
#19I'm trying to understand why people are spinning up so many k8s clusters that they need a tool to do it for them? I have one. And it's managed. I don't think there's significant cost savings to going unmanaged, but maybe. Even so, why would I need a ton of them?
[deleted]
Re: K3sup – bootstrap K3s over SSH in < 60s
#20You can pretty install it without ssh under 60s. The fun starts after it has been installed. We have been running into lot of issues at production with k3s. There I embarked on journey to writing a kubernetes compliant and equivalent platform in rust with the help of claude [1]. It is a fun little project for now, still figuring out stuff, idea is to keep it minimal and single binary every embedded including CNI, and…
Very interesting!
Architecturally - where do you run Postgres ? I assume it would be external to the cluster ? (doing it internally would create a circular dependency ?)