Bare-Metal Kubernetes with K3s

#1

Post by alexellisuk » Tue, Dec 22, 2020, 12:50 PM UTC

Bare-Metal Kubernetes with K3s
blog.alexellis.io

Re: Bare-Metal Kubernetes with K3s

#2

Post by ncrmro » Tue, Dec 22, 2020, 1:12 PM UTC

I actually just got a NUC with 64gb ram. Alpine installed to encrypted NVMe zfs zpool. This runs KVM

One of these guests is alpine with k3s (tried k3os, very limiting in a good way) that allows me to pass a host directory directly into the vm using p9 (tried nfs but lil heavy)

So any storage needs get the benifit of regular snapshots, compression and sync to nas.

Really wish it would be more know that k8 single node is all most people need to get started!

Actual kubernetes deployments and services etc deployed with ansible managed helm

Re: Bare-Metal Kubernetes with K3s

#3

Post by kosikond » Tue, Dec 22, 2020, 1:20 PM UTC

thanks Alex for all your work in k8s ecosystem!

i have 3 cheapish VPS with public IPs and no VPC . Am but struggling to find a way to get k3s HA working with MetalLB, I dont have EIP /NodeBalancer resource, and dont want to resort to mere 1master+2agents cluster. Any tips or links are appreciated

Re: Bare-Metal Kubernetes with K3s

#4

Post by alexellisuk » Tue, Dec 22, 2020, 1:25 PM UTC

kosikond wrote: post #3

thanks Alex for all your work in k8s ecosystem! i have 3 cheapish VPS with public IPs and no VPC . Am but struggling to find a way to get k3s HA working with MetalLB, I dont have EIP /NodeBalancer resource, and dont want to resort to mere 1master+2agents cluster. Any tips or links are appreciated

Are you able to use BGP or ARP as suggested in the post? Does your VPS provide have a managed LoadBalancer option?

I would probably suggest you go for 3x servers in HA, it uses marginally more memory but can tolerate a failure.

You can always go and use DigitalOcean managed K8s, or self-host at home using inlets that I mention in the post.

Re: Bare-Metal Kubernetes with K3s

#5

Post by tasqa » Tue, Dec 22, 2020, 1:25 PM UTC

What I'd like to see for change is actually doing the bare metal part itself. I've seen so many k8s showcase posts of this or that, but never actually someone who's running it on actual servers they own and without using any big four cloud API's (I consider Equinix to be part of those too soon...) to handle the LB/Ingress/Network virtualization stuff they provide and still say it is easy to use..

Re: Bare-Metal Kubernetes with K3s

#6

Post by alexellisuk » Tue, Dec 22, 2020, 1:26 PM UTC

ncrmro wrote: post #2

I actually just got a NUC with 64gb ram. Alpine installed to encrypted NVMe zfs zpool. This runs KVM One of these guests is alpine with k3s (tried k3os, very limiting in a good way) that allows me to pass a host directory directly into the vm using p9 (tried nfs but lil heavy) So any storage needs get the benifit of regular snapshots, compression and sync to nas. Really wish it would be more know that k8 single node…

Maybe you could set up multiple K3s hosts and form a cluster with the VMs? Have you taken a look at inlets-operator that I mention in the post for hosting? My favourite party trick is exposing the IngressController's port 80 and 443. https://docs.inlets.dev/#/get-started/quickstart-ingresscont...

There was a user who was paying 7 USD / mo per site to host 20 side-project sides.. expensive. They switched out to a computer under their desk and saved a lot of money that way.

Re: Bare-Metal Kubernetes with K3s

#7

Post by alexellisuk » Tue, Dec 22, 2020, 1:28 PM UTC

tasqa wrote: post #5

What I'd like to see for change is actually doing the bare metal part itself. I've seen so many k8s showcase posts of this or that, but never actually someone who's running it on actual servers they own and without using any big four cloud API's (I consider Equinix to be part of those too soon...) to handle the LB/Ingress/Network virtualization stuff they provide and still say it is easy to use..

You may enjoy the parts on ARP at the end of the post. I am planning a post on HA K3s with etcd on netbooted RPis. The netbooted RPi part is already available for free to my GitHub Sponsors. Here's a gist I put together whilst figuring out how it should look: https://gist.github.com/alexellis/09b708a8ddeeb1aa07ec276cd8...

Not sure what you mean re: "network virtualisation" though?

Re: Bare-Metal Kubernetes with K3s

#8

Post by rabbut » Tue, Dec 22, 2020, 1:29 PM UTC

I saw Terraform and was out, because I don’t have the money to pour into recurring costs.

Can we have nice things without it being a hook to buy something?

Re: Bare-Metal Kubernetes with K3s

#9

Post by meddlepal » Tue, Dec 22, 2020, 1:33 PM UTC

rabbut wrote: post #8

I saw Terraform and was out, because I don’t have the money to pour into recurring costs. Can we have nice things without it being a hook to buy something?

Terraform is free and opensource?

Or are you making a metacomment about Terraform being painful at times?

Re: Bare-Metal Kubernetes with K3s

#10

Post by alexellisuk » Tue, Dec 22, 2020, 1:33 PM UTC

rabbut wrote: post #8

I saw Terraform and was out, because I don’t have the money to pour into recurring costs. Can we have nice things without it being a hook to buy something?

How is using terraform something to do with spending money?

If you want to go cheap, you can host at home on old servers or RPis.. there's even links included to help you with that.

https://blog.alexellis.io/test-drive-k3s-on-raspberry-pi/