Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bare-Metal Kubernetes with K3s

blog.alexellis.io

Post reply on HN
Print view View original

121–130 of 136 posts

Re: Bare-Metal Kubernetes with K3s

#121

Post by aprdm »

ninjha wrote: post #89
nicolast wrote: post #85

Earlier quoted context omitted.

> Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. The tricky part is long term maintenance and storage. This times 100. Deploying basic clusters is easy. Keeping a test/dev-cluster running for a while? Sure. Keeping production clusters running (TLS cert TTLs expiring, anyone?), upgrading to new K8s versions, proper monitoring (the whole stack, not just your app or the contr…

I’m working on this right now. My theory is that having every cluster object defined in git (but with clever use of third party helm charts to reduce maintenance burden) is the way to go. Our cluster configuration is public[1] and I’m almost done with a blog post going over all the different choices you can make wrt the surrounding monitoring/etc infrastructure on a Kubernetes cluster. [1] https://github.com/ocf/kube…

This looks really cool. Would you include dev apps in the mono repo too? Say you have 40 services by 20 different teams.

Re: Bare-Metal Kubernetes with K3s

#122

Post by hinkley »

cassianoleal wrote: post #16
im_down_w_otp wrote: post #12

I don't entirely understand what's happening here. The title and the post talk a lot about "Bare-Metal", but it also seems to indicate that everything is hosted on servers running Ubuntu. Which, if accurate, seems to be Kubernetes running on Linux, not bare-metal?

Kubernetes relies on containers, which are made out of cgroups and namespaces. Those are Linux features, so there's no way around that. This is as bare-metal as it gets since it's not running on a VM.

Should we unpack these statements though?

How much of linux isn't about cgroups and namespaces? Docker I believe needs about a 100 system calls to get containers to work. How much of the tree could you shake and still have containers work? Would you still call the host Linux, or something else? And what would that system do for Windows and OS X users? Anything?

Could you maintain this 'something else' as a permanent fork, a la Red Hat?

Re: Bare-Metal Kubernetes with K3s

#123

Post by aprdm »

INTPenis wrote: post #103
mrweasel wrote: post #59

Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. The tricky part is long term maintenance and storage. I'm not really sure what people expect gain from these kinds of article, they're great as notes, but it's not something I'd use as a starting point for installing a production Kubernetes cluster. The initial setup of a Kubernetes cluster is something most HN readers could d…

I was one of those backwards people who opposed containers, until late 2019. My start was the official kubernetes docs, step by step. Try everything, write it down, make ansible playbooks. Even used some of their interactive training modules, but I quickly had my own cluster up in vagrant so didn't really need the online shell. Now we have two clusters at work, I have ansible playbooks I'm really happy with that help…

Are they OSS by any chance? And what kind of tasks do they do?

Re: Bare-Metal Kubernetes with K3s

#124

Post by INTPenis »

aprdm wrote: post #123
INTPenis wrote: post #103

Earlier quoted context omitted.

I was one of those backwards people who opposed containers, until late 2019. My start was the official kubernetes docs, step by step. Try everything, write it down, make ansible playbooks. Even used some of their interactive training modules, but I quickly had my own cluster up in vagrant so didn't really need the online shell. Now we have two clusters at work, I have ansible playbooks I'm really happy with that help…

Are they OSS by any chance? And what kind of tasks do they do?

Yes but I won't link them.

They are focused only on CentOS 7, you might be able to find them yourself. As far as I can tell they include everything except persistent storage and HA control plane.

Re: Bare-Metal Kubernetes with K3s

#125

Post by notretarded »

INTPenis wrote: post #103
mrweasel wrote: post #59

Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. The tricky part is long term maintenance and storage. I'm not really sure what people expect gain from these kinds of article, they're great as notes, but it's not something I'd use as a starting point for installing a production Kubernetes cluster. The initial setup of a Kubernetes cluster is something most HN readers could d…

I was one of those backwards people who opposed containers, until late 2019. My start was the official kubernetes docs, step by step. Try everything, write it down, make ansible playbooks. Even used some of their interactive training modules, but I quickly had my own cluster up in vagrant so didn't really need the online shell. Now we have two clusters at work, I have ansible playbooks I'm really happy with that help…

>Now we have two clusters at work

Running your two tightly coupled distributed monoliths of your front and back office systems no doubt!

Re: Bare-Metal Kubernetes with K3s

#126

Post by INTPenis »

notretarded wrote: post #125
INTPenis wrote: post #103

Earlier quoted context omitted.

I was one of those backwards people who opposed containers, until late 2019. My start was the official kubernetes docs, step by step. Try everything, write it down, make ansible playbooks. Even used some of their interactive training modules, but I quickly had my own cluster up in vagrant so didn't really need the online shell. Now we have two clusters at work, I have ansible playbooks I'm really happy with that help…

>Now we have two clusters at work Running your two tightly coupled distributed monoliths of your front and back office systems no doubt!

Heh no single project, staging and prod. In fact prod isn't ready, that's when I need to start learning HA control plane.

Re: Bare-Metal Kubernetes with K3s

#127

Post by tene »

Already__Taken wrote: post #80
Spivak wrote: post #63

Earlier quoted context omitted.

When people talk about “bare metal” k8s they mean on-prem without the support of integrated cloud infrastructure. All the external stuff like load balancers, ingress controllers, routing, bgp, public ip pools, shared storage, vm creation for scaling, are all things the cluster can’t do for itself and have to be implemented and integrated.

Shared storage being a particular head-f to get solved. Everything else is a reasonably approachable problem you can tackle as it comes.

https://rook.io/ is a really great way to get shared storage up quickly and simply.

Re: Bare-Metal Kubernetes with K3s

#128

Post by sandGorgon »

venkyk wrote: post #64
sandGorgon wrote: post #41

Earlier quoted context omitted.

we scale up to about 100 machines. We use spot instances EXTENSIVELY. And that configuration was tricky actually. Its been a couple of months now. Works pretty ok. k3s is actually pretty simple to use now. the tricky part was to integrate with https://github.com/kubernetes/cloud-provider-aws and https://github.com/DirectXMan12/k8s-prometheus-adapter The hardest part is to get it to work with spot instances. we use ht…

Wow! Impressive... what are the advantages over EKS? Just cost benefits or others as well?

We want to have the exact same stack running on laptops and cloud. That's the main goal. Everything else is secondary.

Re: Bare-Metal Kubernetes with K3s

#129

Post by rmdashrfstar »

aprdm wrote: post #123
INTPenis wrote: post #103

Earlier quoted context omitted.

I was one of those backwards people who opposed containers, until late 2019. My start was the official kubernetes docs, step by step. Try everything, write it down, make ansible playbooks. Even used some of their interactive training modules, but I quickly had my own cluster up in vagrant so didn't really need the online shell. Now we have two clusters at work, I have ansible playbooks I'm really happy with that help…

Are they OSS by any chance? And what kind of tasks do they do?

Just use Kubespray, not much sense in wasting time writing your own unless your intention is specifically learning how to manage cluster setup with stateless Ansible

Re: Bare-Metal Kubernetes with K3s

#130

Post by xaduha »

lifty wrote: post #110
xaduha wrote: post #81

Earlier quoted context omitted.

https://wiki.c2.com/?CloseToTheMetal Think various boards bootloaders, not even BIOS to rely on, that's running on bare metal. Calling something running on a normal x64 OS launched with UEFI as such is silly and inaccurate.

I am a professional in the field for many years and your version of the term is not widely accepted. There is a wikipedia page which describes the accepted definition: https://en.wikipedia.org/wiki/Bare-metal_server

Man you guys are laughable, brandishing one wiki page like it's one true source. But nevermind that, several older ones were presented to you and crickets.

https://en.wikipedia.org/wiki/Close_to_Metal

https://en.wikipedia.org/wiki/Bare_machine

https://wiki.c2.com/?CloseToTheMetal

This term goes back to early 2000 at least.

Post reply on HN