Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bare-Metal Kubernetes with K3s

blog.alexellis.io

Post reply on HN
Print view View original

111–120 of 136 posts

Re: Bare-Metal Kubernetes with K3s

#111

Post by arghwhat »

darkwater wrote: post #102
arghwhat wrote: post #37

Earlier quoted context omitted.

See https://en.wikipedia.org/wiki/Bare_machine . Quoting the important part: > In computer science, bare machine (or bare metal) refers to a computer executing instructions directly on logic hardware without an intervening operating system. Bare-metal server has indeed been used to refer to non-virtualized servers, but it's a misnomer. The current terminology for this is "dedicated server".

A misnomer repeated a thousand times by a thousand different people becomes a/the new meaning. It's how human languages work and evolve or change over time. Today when most tech people talk about bare metal they refer to a server that is not virtual.

It is still insignificant compared to the original terminology, which is heavily used in hardware/embedded to distinguish between the use of an OS (embedded RTOS, Linux, etc.) vs. direct programming.

Re: Bare-Metal Kubernetes with K3s

#112

Post by arghwhat »

AntiImperialis2 wrote: post #106
arghwhat wrote: post #39

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Bare_machine > In computer science, bare machine (or bare metal) refers to a computer executing instructions directly on logic hardware without an intervening operating system.

https://en.wikipedia.org/wiki/Bare-metal_server

Yes, I mentioned elsewhere that this has been used to refer to dedicated servers, but it's a marketing misnomer.

Re: Bare-Metal Kubernetes with K3s

#113

Post by arghwhat »

dividedbyzero wrote: post #66
arghwhat wrote: post #37

Earlier quoted context omitted.

See https://en.wikipedia.org/wiki/Bare_machine . Quoting the important part: > In computer science, bare machine (or bare metal) refers to a computer executing instructions directly on logic hardware without an intervening operating system. Bare-metal server has indeed been used to refer to non-virtualized servers, but it's a misnomer. The current terminology for this is "dedicated server".

I've never heard of that before. Besides, dedicated server is an unfortunate term in practice either; I may pay Hetzner for a bunch of dedicated servers, some run a bunch of VMs, some run Kafka without a virtualization layer. My services in containers on Kubernetes in VMs on dedicated servers are still running on dedicated servers, but definitely not what I'd call bare metal. In my experience, calling the Kafka serve…

I think you'll find that hardware/embedded development is not a niche industry.

Re: Bare-Metal Kubernetes with K3s

#114

Post by verdverm »

nouney wrote: post #105
mrweasel wrote: post #72

Earlier quoted context omitted.

>For example, you're no longer manually editing nginx conf, You now have to do the configuration in YAML, which is MUCH MUCH worse.

I don't get it, how do you configure nginx with a YAML file?

You don't directly. You add some extra lines of yaml to an k8s ingress resource, nginx detects this and updates itself. A single nginx container can service many ingresses for each of your apps. The idea is to distribute the config to the app manifests they are related to

Re: Bare-Metal Kubernetes with K3s

#115

Post by throwaway201103 »

mrweasel wrote: post #59

Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. The tricky part is long term maintenance and storage. I'm not really sure what people expect gain from these kinds of article, they're great as notes, but it's not something I'd use as a starting point for installing a production Kubernetes cluster. The initial setup of a Kubernetes cluster is something most HN readers could d…

> Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do.

Care to share an "easy" recipe, because I haven't found one that actually works. It always falls apart for me with the networking.

Let's say I have a cluster of 8 physical nodes and a management node available in my data center and I want to set up k8s for use by my internal users. I'm a solo admin with responsibilty for ~250 physical servers so any ongoing management necessary will be very much a task among many.

Is this blog post a good guide? Is there a better one?

Re: Bare-Metal Kubernetes with K3s

#116

Post by jedifans »

systemvoltage wrote: post #109
ericbarrett wrote: post #91

Earlier quoted context omitted.

Orchestration systems like Kubernetes, Amazon ECS, and (formerly, it’s pretty dead) Mesos run on a cluster of systems and coordinate over the network. What they’re coordinating is running your containerized services, for example your API server, micro services, etc. So you can tell the orchestration service “run 5 copies of this and listen externally on port 9000. When a request comes in on external_ip:9000, redirect…

Thank you. Is there some sort of a PID-feedback-loop control that monitors the CPU/memory load and helps spin up more instances if it sees more traffic? If Kubernetes doesn't do that, what piece of software can help automatically scale if there is a huge traffic spike? Load balance AFAIK doesn't do that. It just helps distribute the load.

Using a HorizontalPodAutoscaler [1] you can scale up and down the amount of "pods" (a grouping of containers that are scheduled as a unit) based on the desired metric.

[1]: https://kubernetes.io/docs/tasks/run-application/horizontal-...

Re: Bare-Metal Kubernetes with K3s

#117

Post by tyldum »

systemvoltage wrote: post #109
ericbarrett wrote: post #91

Earlier quoted context omitted.

Orchestration systems like Kubernetes, Amazon ECS, and (formerly, it’s pretty dead) Mesos run on a cluster of systems and coordinate over the network. What they’re coordinating is running your containerized services, for example your API server, micro services, etc. So you can tell the orchestration service “run 5 copies of this and listen externally on port 9000. When a request comes in on external_ip:9000, redirect…

Thank you. Is there some sort of a PID-feedback-loop control that monitors the CPU/memory load and helps spin up more instances if it sees more traffic? If Kubernetes doesn't do that, what piece of software can help automatically scale if there is a huge traffic spike? Load balance AFAIK doesn't do that. It just helps distribute the load.

Autoscaling is a field of its own, yes. Lots of footguns, though.

Re: Bare-Metal Kubernetes with K3s

#118

Post by nicolast »

ninjha wrote: post #89
nicolast wrote: post #85

Earlier quoted context omitted.

> Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. The tricky part is long term maintenance and storage. This times 100. Deploying basic clusters is easy. Keeping a test/dev-cluster running for a while? Sure. Keeping production clusters running (TLS cert TTLs expiring, anyone?), upgrading to new K8s versions, proper monitoring (the whole stack, not just your app or the contr…

I’m working on this right now. My theory is that having every cluster object defined in git (but with clever use of third party helm charts to reduce maintenance burden) is the way to go. Our cluster configuration is public[1] and I’m almost done with a blog post going over all the different choices you can make wrt the surrounding monitoring/etc infrastructure on a Kubernetes cluster. [1] https://github.com/ocf/kube…

My comment above did not come out of the blue, but based on real-world experience ;-) You may be interested in our MetalK8s project [1] which seems related to yours.

[1] https://github.com/scality/metalk8s

Re: Bare-Metal Kubernetes with K3s

#119

Post by nicolast »

merb wrote: post #101
nicolast wrote: post #85

Earlier quoted context omitted.

> Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. The tricky part is long term maintenance and storage. This times 100. Deploying basic clusters is easy. Keeping a test/dev-cluster running for a while? Sure. Keeping production clusters running (TLS cert TTLs expiring, anyone?), upgrading to new K8s versions, proper monitoring (the whole stack, not just your app or the contr…

first of all with k3s keeping a production cluster running is still pretty easy. second you should always be ready to start from scratch, which is also pretty simple, because of terraform. a lot of people are scared of k8s but they did not even try. they prefer to maintain their scary ansible/puppet whatever script that works only half as good as k8s.

> first of all with k3s keeping a production cluster running is still pretty easy.

Fair enough. I'll admit I have no direct experience with K3s. There are, however, many K8s deployment systems out there which I would not consider 'production-ready' at all even though they're marketed that way.

> second you should always be ready to start from scratch, which is also pretty simple, because of terraform.

That may all be possible if your environment can be spawned using Terraform (e.g., cloud/VMWare environments and similar). If your deployment targets physical servers in enterprise datacenters where you don't even fully own the OS layer, Terraform won't bring much.

> a lot of people are scared of k8s but they did not even try. they prefer to maintain their scary ansible/puppet whatever script that works only half as good as k8s.

We've been deploying and running K8s as part of our on-premises storage product offering since 2018, so 'scared' and 'didn't try' seems not applicable to my experience. Yes, our solution (MetalK8s, it's open source, PTAL) uses a tech 'half as good' as K8s (SaltStack, not Ansible or Puppet) because you need something to deploy/lifecycle said cluster. Once the basic K8s cluster is up, we run as much as possible 'inside' K8s. But IMO K8s is only a partial replacement for technologies like SaltStack and Ansible, i.e., in environments where you can somehow 'get' a (managed) K8s cluster out of thin air.

Re: Bare-Metal Kubernetes with K3s

#120

Post by nicolast »

throwaway201103 wrote: post #115
mrweasel wrote: post #59

Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. The tricky part is long term maintenance and storage. I'm not really sure what people expect gain from these kinds of article, they're great as notes, but it's not something I'd use as a starting point for installing a production Kubernetes cluster. The initial setup of a Kubernetes cluster is something most HN readers could d…

> Getting Kubernetes up and running isn't really the issue anymore, that's pretty easy to do. Care to share an "easy" recipe, because I haven't found one that actually works. It always falls apart for me with the networking. Let's say I have a cluster of 8 physical nodes and a management node available in my data center and I want to set up k8s for use by my internal users. I'm a solo admin with responsibilty for ~25…

An 'easy' way to deploy a cluster could be using kubeadm. Then you'll need a CNI like Calico to get Pod networking up-and-running. However, you'll want to install a bunch of other software on said cluster to monitor it, manage logs,...

Given you're running on physical infrastructure, MetalK8s [1] could be of interest (full disclosure: I'm one of the leads of said project, which is fully open-source and used as part of our commercial enterprise storage products)

[1] https://github.com/scality/metalk8s

Post reply on HN