Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bare-Metal Kubernetes with K3s

blog.alexellis.io

Post reply on HN
Print view View original

21–30 of 136 posts

Re: Bare-Metal Kubernetes with K3s

#21

Post by yokaze »

tasqa wrote: post #5

What I'd like to see for change is actually doing the bare metal part itself. I've seen so many k8s showcase posts of this or that, but never actually someone who's running it on actual servers they own and without using any big four cloud API's (I consider Equinix to be part of those too soon...) to handle the LB/Ingress/Network virtualization stuff they provide and still say it is easy to use..

Where do you see the difficulty?

I've installed k8s with ansible on baremetal (kubespray), more or less just followed the steps here: https://kubernetes.io/docs/setup/production-environment/tool...

No network virtualisation, just Calico. Announce the service ips via BGP from each node running the service and ECMP gives you a (poor mans) load-balancing. Ingress gets such a service-ip. I used simply nginx.

Important here though is, that the router needs to be able to do resilient hashing: Removing a node or adding a node otherwise causes a rehash of all connections leading to breaking connections.

Re: Bare-Metal Kubernetes with K3s

#23

Post by oblio »

scaladev wrote: post #13
im_down_w_otp wrote: post #12

I don't entirely understand what's happening here. The title and the post talk a lot about "Bare-Metal", but it also seems to indicate that everything is hosted on servers running Ubuntu. Which, if accurate, seems to be Kubernetes running on Linux, not bare-metal?

I guess "bare-metal" is Newspeak for "running directly on hardware without a hypervisor".

It's not "Newspeak". Kubernetes is an application, it runs on an OS. You can call it "Newspeak" if you consider anything after VMs became super popular "Newspeak" (around 2010, I think?).

Bare-metal = no VMs or other virtualization involved.

Re: Bare-Metal Kubernetes with K3s

#24

Post by oblio »

_jal wrote: post #19
AntiImperialis2 wrote: post #15

Earlier quoted context omitted.

Bare-metal refers to running on hardware directly (vs. running on VMs or the cloud). I'm interested to know what you thought it was.

In the dark ages, when running an OS in the way you're referring to 'bare metal' was the default, it meant running something without a traditional kernel underneath it. Not many things are written to do that, of course. Oracle used to offer an installation mode like this. It was generally a gimmick - you pay for a tiny bit of performance with a ton of flexibility. There are probably use cases where it makes sense, bu…

Yeah, but it was a super niche term back then. After 2010, at least, bare metal was used a lot more for: "running the OS we need (+ app) directly on a physical server".

This usage has been, in my experience, a lot more widespread.

Re: Bare-Metal Kubernetes with K3s

#25

Post by LoSboccacc »

tasqa wrote: post #5

What I'd like to see for change is actually doing the bare metal part itself. I've seen so many k8s showcase posts of this or that, but never actually someone who's running it on actual servers they own and without using any big four cloud API's (I consider Equinix to be part of those too soon...) to handle the LB/Ingress/Network virtualization stuff they provide and still say it is easy to use..

same, I tried for my test environment server but there were too many undocumented configuration steps, eventually just went with a single node minikube and that was that, I'd love an article with all the kinks worked out.

Re: Bare-Metal Kubernetes with K3s

#26

Post by ClumsyPilot »

ncrmro wrote: post #2

I actually just got a NUC with 64gb ram. Alpine installed to encrypted NVMe zfs zpool. This runs KVM One of these guests is alpine with k3s (tried k3os, very limiting in a good way) that allows me to pass a host directory directly into the vm using p9 (tried nfs but lil heavy) So any storage needs get the benifit of regular snapshots, compression and sync to nas. Really wish it would be more know that k8 single node…

I think this is accurate, but thrre is great educational value when you can see several physical computers working in a cluster, scaling pod on CPU, etc.

I used to have 3 low-power Nuc-style computers, and after playing around for a few months I did the same thing as you have - replaced them with a single beefier machine, and its a lot more practical.

Re: Bare-Metal Kubernetes with K3s

#27

Post by polskibus »

sandGorgon wrote: post #20
tasqa wrote: post #5

What I'd like to see for change is actually doing the bare metal part itself. I've seen so many k8s showcase posts of this or that, but never actually someone who's running it on actual servers they own and without using any big four cloud API's (I consider Equinix to be part of those too soon...) to handle the LB/Ingress/Network virtualization stuff they provide and still say it is easy to use..

we are running k3s on metal in production. works great actually. we use haproxy as ingress and lb.

How stable do you see it ? What's your cluster size? How long has it been running? Any tips for how to approach starting such setup?

Re: Bare-Metal Kubernetes with K3s

#28

Post by aetimmes »

tasqa wrote: post #5

What I'd like to see for change is actually doing the bare metal part itself. I've seen so many k8s showcase posts of this or that, but never actually someone who's running it on actual servers they own and without using any big four cloud API's (I consider Equinix to be part of those too soon...) to handle the LB/Ingress/Network virtualization stuff they provide and still say it is easy to use..

I recently set up a homelab of ~10 k3os+k3s nodes on NUCs. Setting up MetalLB on top of the base k3s installation made exposing services on their own IP addresses pretty dead simple.

Re: Bare-Metal Kubernetes with K3s

#29

Post by cassianoleal »

xaduha wrote: post #18
cassianoleal wrote: post #16

Earlier quoted context omitted.

Kubernetes relies on containers, which are made out of cgroups and namespaces. Those are Linux features, so there's no way around that. This is as bare-metal as it gets since it's not running on a VM.

Point here is that term bare-metal cannot apply to applications relying on OS features. Unikernels are the closest thing you can use this term in this area.

Wikipedia [0] doesn't agree with you (neither do I, but who am I anyway?):

"A bare-metal server is a computer server that hosts one tenant, or consumer, only.[1] The term is used for distinguishing between servers that can host multiple tenants and which utilize virtualisation and cloud hosting.[2] Such servers are used by a single consumer and are not shared between consumers. Each server may run any amount of work for a user, or have multiple simultaneous users, but they are dedicated entirely to the entity who is renting them. Unlike servers in a data centre, they are not being shared between multiple customers.

Bare-metal servers are physical servers. Each server offered for rental is a distinct physical piece of hardware that is a functional server on its own. They are not virtual servers running in multiple pieces of shared hardware."

[0] https://en.wikipedia.org/wiki/Bare-metal_server

Re: Bare-Metal Kubernetes with K3s

#30

Post by aij »

scaladev wrote: post #13
im_down_w_otp wrote: post #12

I don't entirely understand what's happening here. The title and the post talk a lot about "Bare-Metal", but it also seems to indicate that everything is hosted on servers running Ubuntu. Which, if accurate, seems to be Kubernetes running on Linux, not bare-metal?

I guess "bare-metal" is Newspeak for "running directly on hardware without a hypervisor".

You mean it's Newspeak for "running under a supervisor, but without the supervisor running under a hypervisor".
Post reply on HN