Live data from Hacker News

Kubernetes on Hetzner: cutting my infra bill by 75%

bilbof.com

91–100 of 229 posts

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#91
post #15

When I worked in web hosting (more than 10 years ago), we would constantly be blackholeing Hetzner IPs due to bad behavior. Same with every other budget/cheap vm provider. For us, it had nothing to do with geo databases, just behavior. You get what you pay for, and all that.

They could put the backend on Hetzner, if it makes sense (for example queues or batch processors).

[deleted]

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#92

I have experience running Kubernetes clusters on Hetzner dedicated servers, as well as working with a range of fully or highly managed services like Aurora, S3, and ECS Fargate. From my experience, the cloud bill on Hetzner can sometimes be as low as 20% of an equivalent AWS bill. However, this cost advantage comes with significant trade-offs. On Kubernetes with Hetzner, we managed a Ceph cluster using NVMe storage,…

I've never operated a kubernetes cluster except for a toy dev cluster for reproducing support issues. One day it broke because of something to do with certificates (not that it was easy to determine the underlying problem). There was plenty of information online about which incantations were necessary to get it working again, but instead I nuked it from orbit and rebuilt the cluster. From then on I did this every few…

That's just not how it works on any scale other than "toy"

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#93

I have experience running Kubernetes clusters on Hetzner dedicated servers, as well as working with a range of fully or highly managed services like Aurora, S3, and ECS Fargate. From my experience, the cloud bill on Hetzner can sometimes be as low as 20% of an equivalent AWS bill. However, this cost advantage comes with significant trade-offs. On Kubernetes with Hetzner, we managed a Ceph cluster using NVMe storage,…

[dead]

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#94
post #49
post #41

Earlier quoted context omitted.

Reading comments from the past few days makes it seem like dealing with Hetzner is a pain (and as far as I can tell, they aren't really that cheaper than the competitors).

What competitors are similar to Hetzner in pricing? Last I checked, they seemed quite a bit cheaper than most.

OVH is larger provider, servers usually not significantly more expensive than hetzner.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#95

Earlier quoted context omitted.

I've never operated a kubernetes cluster except for a toy dev cluster for reproducing support issues. One day it broke because of something to do with certificates (not that it was easy to determine the underlying problem). There was plenty of information online about which incantations were necessary to get it working again, but instead I nuked it from orbit and rebuilt the cluster. From then on I did this every few…

That's just not how it works on any scale other than "toy"

Right, but certs get out of date unless somebody does something about it, that was my point.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#96

> Hetzner volumes are, in my experience, too slow for a production database. While you may in the past have had a good experience running customer-facing databases on AWS EBS, with Hetzner's volumes we were seeing >50ms of IOWAIT with very low IOPS. There is a surprisingly easy way to address this issue: use (ridiculously cheap) Hetzner metal machines as nodes. The ones with nvme storage offer excellent performance f…

Thanks for the Pigsty link. I have been a big fan of running Postgres on metal machines.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#97

Earlier quoted context omitted.

> Has anyone ever toyed around with the idea? Sidero Omni have done this: https://omni.siderolabs.com They run a Wireguard network between the nodes so you can have a mix of on-premise and cloud within one cluster. Works really well but unfortunately is a commercial product with a pricing model that is a little inflexible. But at least it shows it's technically possible so maybe open source options exist.

You could make a mesh with something like Netmaker to achieve similar using FOSS. Note I haven’t used Netmaker in years but I was able to achieve this in some of their earlier releases. I found it to be a bit buggy and unstable at the time due to it being such young software but it may have matured enough now that it could work in an enterprise grade setup. The sibling comments recommendation, Nebula, does something…

[deleted]

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#98

I loved the article. Insightful, and packed with real world applications. What a gem. I have a side-question pertaining to cost-cutting with Kubernetes. I've been musing over the idea of setting up Kubernetes clusters similar to these ones but mixing on-premises nodes with nodes from the cloud provider. The setup would be something like: - vCPUs for bursty workloads, - bare metal nodes for the performance-oriented wo…

> Has anyone ever toyed around with the idea? Sidero Omni have done this: https://omni.siderolabs.com They run a Wireguard network between the nodes so you can have a mix of on-premise and cloud within one cluster. Works really well but unfortunately is a commercial product with a pricing model that is a little inflexible. But at least it shows it's technically possible so maybe open source options exist.

I believe the Cilium CNI has this functionality built in. Other CNIs may do also.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#99
post #45

Earlier quoted context omitted.

Quicker to type and scan! Though I admit this is preference, delimiters would work fine too. Parsing works the same but is based on a simple regex rather than splitting on a hyphen. euc=eu central; 1=zone/dc; p=production; wkr=worker; 1=node id

Thanks for getting back to me! Now that you've written it out, it's plainly obvious, but for me the readability and flexibility of delimiters beats the speed of typing and scanning. Many a times I've been grateful that I added delimiters because then I was no longer be hamstrung by any potential changes to the length of any particular segment within the name.

You can more easily double-click-select the full hostname when there are no delimiters.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#100
post #72

Earlier quoted context omitted.

i recently read an article about running k8s on the oracle free tier and was looking to try it. i'm curious, are there any specific pain points that are making you think of switching?

Nope, just Oracle being a corp with a nasty reputation. Honesty it was easy to set up and has been super stable, and if you go ARM the amount of resources you get for free is crazy. I actually do recommend it for personal projects on the like. I'd just be hesitant about building a business based on any Oracle offering.

Got it, thanks for the clarification! I’ll be using it for a personal project so that sounds great.
Post reply on HN