A bit off topic, but has anyone used K8s along with Proxmox?
Yes. I run my k8s cluster on a Proxmox host. Any specific questions you have?
I'm mostly curious how you got it all set up? Did you install something like CoreOS as the host?
K3s – Lightweight Kubernetes
161–170 of 194 posts
Re: K3s – Lightweight Kubernetes
#162A bit off topic, but has anyone used K8s along with Proxmox?
Not a k8s user, but no reason it shouldn't work from what I read about it. Spawn a few CoreOS or RancherOS VMs and off you go. Some people even used it with LXC, though it seems a bit of a work. e.g. https://medium.com/@kvaps/run-kubernetes-in-lxc-container-f0...
Gonna check this out, thank you! I tried installing CoreOS a couple years ago and I recall encountering some issues using LXC, but maybe it'll work now :)
Re: K3s – Lightweight Kubernetes
#163Oh nice - this is by the Rancher guys. Would love to compare this not with k8s, but with Docker Swarm. Docker Swarm on Raspberry Pi is a very common thing. So, from a performance perspective, these are on par. Another question is around ingress and network plugin - is it a seamless "batteries included" experience ? Because these are two of the biggest pains in k8s to decide and setup .
It includes the flannel network plugin by default (although you can change this) and a basic service load balancer (technically not an ingress, but providing the same functionality). From the README: > k3s includes a basic service load balancer that uses available host ports. If you try to create a load balancer that listens on port 80, for example, it will try to find a free host in the cluster for port 80. If no po…
Traefik is deployed for layer-7 ingress.
Re: K3s – Lightweight Kubernetes
#164It is nice to see people working on making software smaller!
It leaves me wondering though, what is Kubernetes doing with all of these resources? When I saw "lightweight" in the title I expected 5.12MB of RAM, not 512MB.
Re: K3s – Lightweight Kubernetes
#165Re: K3s – Lightweight Kubernetes
#166Earlier quoted context omitted.
Agreed. Kubernetes (installed via kubeadm) on 1 GB of RAM is a shaky proposition at best; I've been managing a cluster on Raspberry Pis and the master often hits swap managing three Nodes with a dozen or so Pods.
K8s doesn’t support swap, you have to disable swap to even run kubelet.
There is an override flag.
It works, but it kind of invalidates all assumptions k8s makes about free memory on a node
Re: K3s – Lightweight Kubernetes
#167Earlier quoted context omitted.
> this has the same great K8S API, that Swarm lacks. (Deployment is a first class citizen in kube land, but you only have to make do with the service YMLs in the Swarm sphere.) With all due respect, some (like me) consider that a feature of Swarm. If k3s comes with the same API (and implicit yaml complexity) of k8s, but just reduces RAM usage...well that's not very interesting for me. Docker Swarm is very lean - ther…
I use Docker Compose on a small, cloud-hosted VM to run all my development dependencies and stuff I need for testing (I've got RabbitMQ, Postgres and Splunk running at present). It was really simple to setup, and "just works" - and the nodes don't have to compete with a greedy K8s orchestrator for resources!
Ditto. For a single machine you don't need "orchestration" - you just need something cleaner than a bash script to start/stop and potentially restart your containers.
I set up my home server with minikube more as a learning exercise, but went back to docker-compose for running all the things I actually rely on (Plex for instance) just because it's so much simpler.
Re: K3s – Lightweight Kubernetes
#168Earlier quoted context omitted.
Based on the README [0], quickly comparing to Swarm: - this has the same great K8S API, that Swarm lacks. (Deployment is a first class citizen in kube land, but you only have to make do with the service YMLs in the Swarm sphere.) - k3s lacks some in-tree plugins, that swarm might have (mount cloud provider managed block device), but there are out of tree addons - sqlite instead of etcd3 [but available], so out of the…
> great K8S API, that Swarm lacks Hmm... nothing against k8s, but it’s deployment api is an abomination on par with aws cloudformation. You need teams of yaml engineers to manage these things.
Is YAML engineer a real thing? I always considered YAML to be a slightly hacky configuration DSL that I needed to be familiar with, not an actual core, career, competence.
For example, can I become an INI engineer?
Re: K3s – Lightweight Kubernetes
#169This is extremely welcome. Kube needs to glom on to all available architectures to fulfill its destiny, to some extent, and the memory/cpu usage of kubeapi et al. can be prohibitive for small setups. Having to revert to ansible/systemd in "some cases" really weakens the story of a universal datacenter O/S. My hope is for more competition for development k8s (or k3s!) - minikube and docker-for-desktop is plagued with…
I really want to do this as well - we've just started looking at minikube for our group, and I'd love to use k3s instead, but does anyone know if it works on OSX? I was able to get a cluster running on my laptop in under a minute, which is pretty amazing.
Have you seen or used Microk8s ? Seems like it might be what you are looking for. Can be installed and removed via snap package too
Re: K3s – Lightweight Kubernetes
#170Technically speaking: cool Practically speaking: I worry this will become a parallel but subtlety different implementation of Kubernetes with it's own quirks.
> Practically speaking: I worry this will become a parallel but subtlety different implementation of Kubernetes with it's own quirks. Hello, we're already there. Despite the CNCF certification process, there are many flavours of k8s, all with their own quirks. Being on the product side, it is quite painful to support.
I am aware of this problem (getambassador.io dev as well as infra engineer). The differences usually manifest at the edges rather than the internals right now. This has the potential to be even more disruptive problem.
I agree the CNCF certification process isn't comprehensive enough.