Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes on Hetzner: cutting my infra bill by 75%

bilbof.com

Post reply on HN
Print view View original

191–200 of 229 posts

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#191

Post by p_l »

tehlike wrote: post #168
surrTurr wrote: post #110

> Hetzner volumes are, in my experience, too slow for a production database. While you may in the past have had a good experience running customer-facing databases on AWS EBS, with Hetzner's volumes we were seeing >50ms of IOWAIT with very low IOPS. See https://github.com/rook/rook/issues/14999 for benchmarks. I set up rook ceph on a talos k8s cluster (with vm volumes) and experienced similar low performance; however…

SSD volumes are physically on the same node, and afaik not redundant. The cloud vms are ceph clusters behind the scenes, and writes need to commit for 3+ machines. It's both network latency and inherent process latency Additionally, hetzner has an IOPS limit of 5000 and write limit of some amount that does not scale with the size of database. 50G has the same limits as 5TB. For this reason, people are sometimes using…

RAID10 on local SSDs is pretty performant option, but yeah, it's per node.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#192

Post by p_l »

teitoklien wrote: post #70
hipadev23 wrote: post #67

Earlier quoted context omitted.

I went back to AWS. Expensive but reliable and support I can get ahold of. I’d still like to explore OVH someday though.

Nothing beats aws tbh, the level of extra detail aws adds, like emailing and alerting a gazillion times before making any changes to underlying hardware, even if non disruptive. Robust I use OVH because the cost reduction supremely adds up for my workloads (remote video editing/ custom rendering farm at scale with a lot more cheaper OVH s3 suitable for my temporary but too many asset workload with high egress require…

I wonder how out of norm my AWS experience is based on the comments here...

Said experience being that the very highly paid support team ghosted us when we asked questions guided by their docs (AWS Workspace stuff), probably after finding that there are no answers and that we went with service based on promise of feature that apparently didn't exist

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#193

Post by infecto »

p_l wrote: post #189
0xbadcafebee wrote: post #182

I used to do my own car maintenance, because I wanted to save money, and it was fun. It turned out it was more complex than I thought, things slowly fell apart or broke. I spent a good deal of time "re-fixing" things. Spent probably thousands on tools over the years, partly replacing the cheap stuff that broke or rusted quickly. My cars were often up on blocks. But I learned a lot of great lessons. The biggest one? S…

Unless you find out that calling AA for everything and going to big name mechanics garage every time costs you significant portion of your budget. A lot of it is finding balance between what to do yourself, what to outsource, and it's not as easy or clean as some people here like to claim.

+1 to this. I agree with the takeaway that paying for managed Kubernetes is the way to go 9 times out of 10 but I think in life its quite powerful to attempt to do things yourself sometimes. No, you should not go out and buy thousands of dollars worth of tools and do everything yourself but its pretty powerful to be able to do simple things in your life...change oil, replace light switches/sockets. The reason I personally find it powerful is that it helps me quantify what is worth outsourcing and what is not and it also builds mental models for how things work which is important when outsourcing work.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#194

Post by tutfbhuf »

redbell wrote: post #130
tutfbhuf wrote: post #116

Earlier quoted context omitted.

It is my ouput, but I use ChatGPT to fix my spelling and grammar. Maybe my prompt for that should be refined in order to not alter the wording too much.

While using ChatGPT for enhancing your writings is not wrong by any means, reviewing the generated output and re-editing when necessary is essential to avoid robotic writing style that may smell unhuman. For instance, these successive paragraphs: "In contrast, using AWS.." and "In essence, with AWS.." leaves a bad taste in your brain when read consecutively.

I agree with you, failed on that one

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#196

Post by p_l »

lurking_swe wrote: post #86
vachina wrote: post #77

Earlier quoted context omitted.

To be fair what use is a server if you can’t afford to keep it running. This is especially true for very bootstrapped startups.

We all start somewhere. :) Hetzner can be a good fit for many small companies. But let’s also be honest, if you’re THAT bootstrapped, you probably have no business running kubernetes to begin with. If the company has a short runway, it doesn’t make sense to work on a complex architecture from the start. Focus on shipping something and getting revenue.

Depends entirely on what is your starting skillset.

K8s is my tool of choice when I am that boostrapped, because a single server with k3s thrown on it will cost me maybe 80 EUR a month and hold all environments plus CI/CD plus various self-hosted business components (with backup sent over to separate provider, just in case), and I'll be free to build my project instead of messing with server setup or worry about cloud bills.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#197

Post by theappsecguy »

0xbadcafebee wrote: post #182

I used to do my own car maintenance, because I wanted to save money, and it was fun. It turned out it was more complex than I thought, things slowly fell apart or broke. I spent a good deal of time "re-fixing" things. Spent probably thousands on tools over the years, partly replacing the cheap stuff that broke or rusted quickly. My cars were often up on blocks. But I learned a lot of great lessons. The biggest one? S…

I don’t know. My shop wanted 1800 to change my brakes. I bought the parts for 300 and got it done in a day (first time). Seems like a pretty good payback and good skill to have. My neighbour has a car lift which certainly helped.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#198

Post by p_l »

cedws wrote: post #81
MathiasPius wrote: post #47

Earlier quoted context omitted.

I set up a Talos bare metal cluster about a year ago, and documented the whole process on my website. Feel free to reach out if you have any questions!

Any thoughts/feelings about Talos vs Bottlerocket?

Did bottlerocket get any closer to stable and usable outside AWS walled garden?

Last time I tried was admittedly in 2022, but in testing which distro to go with bottlerocket lost because we couldn't setup local builds...

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#199

Post by dotancohen »

DonHopkins wrote: post #186
theshrike79 wrote: post #176

Earlier quoted context omitted.

Unless the spelling and grammar is HORRENDOUS people won't really care. Bad English is the words most used language, we all deal with it every day. Just using your browser's built-in proofreader is enough in 99.9% of the cases. Using ChatGPT to rewrite your ideas will make them feel formulaic (LLMs have a style and people exposed to them will spot it instantly, like a code smell) and usually needlessly verbose.

You can tell it's AI when it refuses to take a side and equivocally considers issues first on one hand and then the other hand, but can't get the number of fingers right. Or as ChatGPT would put it: Precise grammar and spelling are undeniably important, but minor imperfections in English rarely obstruct communication. As the most widely used language in the world, English is highly flexible, and most people navigate…

I could actually hear the different voices in my head as I read the second and third paragraphs, distinct from the first. Your assessment of the unable-to-take-a-side is spot on for OpenAI, possibly Gemini too, but not for all LLMs.

Re: Kubernetes on Hetzner: cutting my infra bill by 75%

#200

Post by wrd83 »

jpgvm wrote: post #150
jonas21 wrote: post #57

This is an interesting writeup, but I feel like it's missing a description of the cluster and the workload that's running on it. How many nodes are there, how much traffic does it receive, what are the uptime and latency requirements? And what's the absolute cost savings? Saving 75% of $100K/mo is very different from saving 75% of $100/mo.

In my experience noone bothers unless they are using GPUs or they are already at 100k/mo. I do think 100k/mo is the tipping point actually, that is $1.2M/yr. It costs around $400k/yr in engineering salaries to reasonably support a sophisticated bare metal deployment (though such people can generally do that AND provide a lot of value elsewhere in the business, so really it's actual cost is lower than this) and about…

If you want to do HA kubernetes, you need oncalls and at least 10 engineers to get a stable rotation.

If you do that in Europe you have to pay them during standby hours.

400k/year seems very low to me.

Post reply on HN