Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bare-Metal Kubernetes, Part I: Talos on Hetzner

datavirke.dk

Post reply on HN
Print view View original

31–40 of 79 posts

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#31

Post by bionsystem »

KronisLV wrote: post #27
wg0 wrote: post #12

I've come to the conclusion (after trying kops, kubespray, kubeadm, kubeone, GKE, EKS) that if you're looking for Docker swarm is to Kubernetes what SQLite is to PostgreSQL. To some extent.

> I've come to the conclusion (after trying kops, kubespray, kubeadm, kubeone, GKE, EKS) that if you're looking for Personally, I'd also consider throwing Portainer in there, which gives you both a nice way to interact with the cluster, as well as things like webhooks: https://www.portainer.io/ With something like Apache, Nginx, Caddy or something else acting as your "ingress" (taking care of TLS, reverse proxy, head…

When I was deploying swarm clusters I would have a default stack.yml file with portainer for admin, traefik for reverse-proxying, and prometheus, grafana, alertmanager, unsee, cadvisor, for monitoring and metrics gathering. All were running on their own docker network completely separated from the app and were only accessible by ops (and dev if requested, but not end users). It was quite easy to deploy with HEAT+ansible or terraform+ansible and the hard part was the ci/cd for every app each in its tenant, but it worked really really well.

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#32

Post by ori_b »

doctorpangloss wrote: post #26
wg0 wrote: post #12

I've come to the conclusion (after trying kops, kubespray, kubeadm, kubeone, GKE, EKS) that if you're looking for Docker swarm is to Kubernetes what SQLite is to PostgreSQL. To some extent.

Any sufficiently complicated Docker Swarm, Heroku, Elastic Beanstalk, Nomad or other program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of vanilla Kubernetes.

Unfortunately, the above statement also applies to kubernetes.

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#34

Post by stavros »

wg0 wrote: post #12

I've come to the conclusion (after trying kops, kubespray, kubeadm, kubeone, GKE, EKS) that if you're looking for Docker swarm is to Kubernetes what SQLite is to PostgreSQL. To some extent.

I was of the same opinion, so I rolled my own thin layer over Compose:

https://harbormaster.readthedocs.io/

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#36

Post by amazingman »

husarcik wrote: post #20
wg0 wrote: post #12

I've come to the conclusion (after trying kops, kubespray, kubeadm, kubeone, GKE, EKS) that if you're looking for Docker swarm is to Kubernetes what SQLite is to PostgreSQL. To some extent.

The docker swarm ecosystem is very poor as far as tooling goes. You're better off using docker-compose (? maybe docker swarm) and then migrating to k3s if you need a cluster. My docker swarm config files are nearly the same craziness as my k3s config files so I figured I might as well benefit from the tooling in Kubernetes. Edit for more random thoughts: being able to use helm to deploy services helped me switch to k…

This is almost exactly my experience with Docker Compose, which is lionized by commenters in nearly every Kubernetes thread I read on HN. It's great and super simple and easy ... until you want to wire multiple applications together, you want to preserve state across workload lifecycles for stateful applications, and/or you need to stand up multiple configurations of the same application. The more you want to run applications that are part of a distributed system, the uglier your compose files get. Indeed, the original elegant Docker Compose syntax just couldn't do a bunch of things and had to be extended.

IMO a sufficiently advanced Docker Compose stack is not appreciably simpler than the Kubernetes manifests would be, and you don't get the benefits of Kubernetes' objects and their controllers because Docker Compose is basically just stringing low-level concepts together with light automation.

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#37

Post by amazingman »

ori_b wrote: post #32
doctorpangloss wrote: post #26

Earlier quoted context omitted.

Any sufficiently complicated Docker Swarm, Heroku, Elastic Beanstalk, Nomad or other program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of vanilla Kubernetes.

Unfortunately, the above statement also applies to kubernetes.

A pithy response to be sure, but is it true? Every Kubernetes object type exists within a well-specified hierarchy, has a well-specified specification, an API version, and documentation. Most of the object families' evolution are managed by a formal SIG. Not sure how any of that qualifies as ad-hoc or informal.

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#38

Post by hn_user82179 »

wg0 wrote: post #12

I've come to the conclusion (after trying kops, kubespray, kubeadm, kubeone, GKE, EKS) that if you're looking for Docker swarm is to Kubernetes what SQLite is to PostgreSQL. To some extent.

I’ve been at a company running swarm in prod for a few years. There have been several nasty bugs that are fun to debug but we’ve accumulated several layers of slapped bandaids trying to handle swarm’s deficiencies. I can’t say I’d pick it again, nor would I recommend it for anyone else.

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#39

Post by mardifoufs »

ori_b wrote: post #32
doctorpangloss wrote: post #26

Earlier quoted context omitted.

Any sufficiently complicated Docker Swarm, Heroku, Elastic Beanstalk, Nomad or other program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of vanilla Kubernetes.

Unfortunately, the above statement also applies to kubernetes.

Kubernetes is anything but adhoc. That's the best thing, but can also be the most annoying, part about it

Re: Bare-Metal Kubernetes, Part I: Talos on Hetzner

#40

Post by ralala »

MathiasPius wrote: post #2

I recently rebuilt my Kubernetes cluster running across three dedicated servers hosted by Hetzner and decided to document the process. It turned into a (so far) 8-part series covering everything from bootstrapping and firewalls to setting up persistent storage with Ceph. Part I: Talos on Hetzner https://datavirke.dk/posts/bare-metal-kubernetes-part-1-talo... Part II: Cilium CNI & Firewalls https://datavirke.dk/posts/…

Interesting read. I have just setup a very similar cluster this week: 3 node bare metal cluster in a 10G mesh network. Decided for Debian, RKE2, Calico and Longhorn. Encryption is done using LUKS FDE. For Load Balancing I am using the HCloud Load Balancer (in TCP mode). At first I had some problems with the mesh network as the CNI would only bind to a single interface. Finally solved it using a bridge, veth and isolated ports.
Post reply on HN