Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes on Hetzner: cutting my infra bill by 75%

bilbof.com

Post reply on HN
Print view View original

71–80 of 229 posts

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#71

Post by jacooper »

ronsor wrote: post #41
hipadev23 wrote: post #19

Be careful with Hetzner, they null routed my game server on launch day due to false positives from their abuse system, and then took 3 days for their support team to re-enable traffic. By that point I had already moved to a different provider of course.

Reading comments from the past few days makes it seem like dealing with Hetzner is a pain (and as far as I can tell, they aren't really that cheaper than the competitors).

Honestly hetzner supoort has bren outstanding from my experience. They are always there and very responsive using email

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#72

Post by Volundr »

maestrae wrote: post #59
Volundr wrote: post #10

I haven't used it personally, but https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne... looks amazing as a way to setup and manage kubernetes on Hetzner. At the moment I'm on Oracle free tier, but I keep thinking about switching to it to get off... Well Oracle.

i recently read an article about running k8s on the oracle free tier and was looking to try it. i'm curious, are there any specific pain points that are making you think of switching?

Nope, just Oracle being a corp with a nasty reputation. Honesty it was easy to set up and has been super stable, and if you go ARM the amount of resources you get for free is crazy. I actually do recommend it for personal projects on the like. I'd just be hesitant about building a business based on any Oracle offering.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#73

Post by no_carrier »

> While DigitalOcean, like other providers, offers a free managed control plane, there is typically a 100% markup on the nodes that belong to these managed clusters.

I don't think this is true. With Digital Ocean, the worker nodes are the same cost as regular droplets, there's no additional costs involved. This makes Digital Ocean's offering very attractive - free control plane you don't have to worry about, free upgrades, and some extra integrations to things like the load balancer, storage, etc. I can't think of a reason to not go with that over self-managed.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#74

Post by czhu12 »

no_carrier wrote: post #73

> While DigitalOcean, like other providers, offers a free managed control plane, there is typically a 100% markup on the nodes that belong to these managed clusters. I don't think this is true. With Digital Ocean, the worker nodes are the same cost as regular droplets, there's no additional costs involved. This makes Digital Ocean's offering very attractive - free control plane you don't have to worry about, free upg…

The actual nodes are still way more expensive on digital ocean than they are in Hetzner. That’s probably the main reason.

8GB RAM, shared cpu on hetzner is ~$10

Equivalent on digital ocean is $48

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#75

Post by stackskipton »

spwa4 wrote: post #68
tutfbhuf wrote: post #56

I have experience running Kubernetes clusters on Hetzner dedicated servers, as well as working with a range of fully or highly managed services like Aurora, S3, and ECS Fargate. From my experience, the cloud bill on Hetzner can sometimes be as low as 20% of an equivalent AWS bill. However, this cost advantage comes with significant trade-offs. On Kubernetes with Hetzner, we managed a Ceph cluster using NVMe storage,…

> Determining which option is more cost-effective requires a thorough TCO (Total Cost of Ownership) analysis. While Hetzner may seem cheaper upfront, the additional hours required for DevOps work can offset those savings. Sure, but the TLDR is going to be that if you employ n or more sysadmins, the cost savings will dominate. With 2 Second if you have a "big" cost, whatever it is, bandwidth, disk space (essentially a…

Not always. Employing Sysadmins doesn't mean Hetzner is cheaper because those "Sysadmin/Ops type people" are being hired to managed the Kubernetes cluster. And Ops type people who truly know Kubernetes are not cheap.

Sure, you can get away with legoing some K3S stuff together for a while but one major outage later, and that cost saving might have entirely disappeared.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#76

Post by stackskipton »

BillFranklin wrote: post #42
usrme wrote: post #39

This is probably out of left field, but what is the benefit of having a naming scheme for nodes without any delimiters? Reading at a glance and not knowing the region name convention of a given provider (i.e. Hetzner), I'm at a loss to quickly decipher the " " to "euc1pmgr1". I feel like I'm missing something because having delimiters would make all sorts of automated parsing much easier.

Quicker to type and scan! Though I admit this is preference, delimiters would work fine too. Parsing works the same but is based on a simple regex rather than splitting on a hyphen. euc=eu central; 1=zone/dc; p=production; wkr=worker; 1=node id

Yea, not putting in delimiter and then us having to change our format has bitten me so many times. Delimiter or bust.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#77

Post by vachina »

lurking_swe wrote: post #50
gurchik wrote: post #48

Earlier quoted context omitted.

> (and as far as I can tell, they aren't really that cheaper than the competitors) Can you say more? Their Cloud instances, for example, are less than half the cost of OVH's, and less than a fifth of the cost of a comparable AWS EC2 instance.

even free servers are of no use if it’s not usable during a product launch. :) You get what you pay for i guess. But i do agree, it is much cheaper.

To be fair what use is a server if you can’t afford to keep it running. This is especially true for very bootstrapped startups.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#78

Post by s3rius »

That's a really good article. Actually, recently we were migrating as well and we were using dedicated nodes in our setup.

In order to integrate a load-balancer provided by hetzner with our k8s on dedicated servers we had to implement a super thin operator that does it: https://github.com/Intreecom/robotlb

If anyone will be inspired by this article and would want to do the same, feel free to use this project.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#79

Post by threeseed »

juiyhtybr wrote: post #27
ffsm8 wrote: post #23

Earlier quoted context omitted.

And if you wanna be lazy, there is a tailscale integration to run the cluster communication over it. https://tailscale.com/kb/1236/kubernetes-operator They've even improved it, so you can now actually resolve the services etc via the tailnet dns https://tailscale.com/learn/managing-access-to-kubernetes-wi... I haven't tried that second part though, only read about it.

yes, like i said, throw an overlay on that motherfucker and ignore the fact that when a customer request enters the network it does so at the cloud provider, then is proxied off to the final destination, possibly with multiple hops along the way. you can't just slap an overlay on and expect everything to work in a reliable and performant manner. yes, it will work for your initial tests, but then shit gets real when y…

a) Not every Kubernetes cluster is customer facing.

b) You should be architecting your platform to accomodate these very common networking scenarios i.e. having edge caching. Because slow backends can be caused by a range of non-networking issues as well.

c) Many cloud providers (even large ones like AWS) are hosted in or have special peering relationships with third party DCs e.g. [1]. So there are no "internet quality links" if you host your equipment in one of the major DCs.

[1] https://www.equinix.com.au/partners/aws

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#80

Post by mountainriver »

tutfbhuf wrote: post #56

I have experience running Kubernetes clusters on Hetzner dedicated servers, as well as working with a range of fully or highly managed services like Aurora, S3, and ECS Fargate. From my experience, the cloud bill on Hetzner can sometimes be as low as 20% of an equivalent AWS bill. However, this cost advantage comes with significant trade-offs. On Kubernetes with Hetzner, we managed a Ceph cluster using NVMe storage,…

Manually updating k8s clusters is a huge tradeoff. I can’t imagine doing that to save a couple bucks unless I was desperate
Post reply on HN