Live data from Hacker News

We cut our Mongo DB costs by 90% by moving to Hetzner

prosopo.io

171–180 of 214 posts

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#171

Why in the world do people choose Mongo over Postgres? I'm legit curious. Is it inexperience? Javascript developers who don't know backend or proper data modeling (or about jsonb)? Is this type of decision coming down from non-technical management? Are VCs telling their portfolio companies what to use so they have something to burn their funding on? It's just really confounding, especially when there's even mongo-api…

We've been using mongodb for the past 8 years. What we like:

- schema-less: we don't have to think about DDL statements at any point.

- oplog and change streams as built-in change data capture.

- it's dead simple to setup a whole new cluster (replica set).

- IMO you don't need a designated DBA to manage tens of replica sets.

- Query language is rather low-level and that makes performance choices explicit.

But I have to admit that our requirements and architecture play to the strength of mongodb. Our domain model is neatly described in a strongly typed language. And we use a sort of event sourcing.

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#172

> Here's how we managed to cut our costs by 90% You could cut your MongoDB costs by 100% by not using it ;) > without sacrificing performance or reliability. You're using a single server in a single datacenter. MongoDB Atlas is deployed to VMs on 2-3 AZs. You don't have close to the same reliability. (I'm also curious why their M40 instance costs $1000, when the Pricing Calculator ( https://www.mongodb.com/pricing )…

How often do the AZs matter? - I feel like there's a major global outage on every cloud provider of choice, at least every other year, yet I don't remember any outage where only a single AZ went down (I'm on AWS).

Fighting said outages is often made harder is that the providers themselves just don't admit to anything being wrong, everything's green on the dashboard yet 4 out of 5 requests are timing out.

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#173

> Here's how we managed to cut our costs by 90% You could cut your MongoDB costs by 100% by not using it ;) > without sacrificing performance or reliability. You're using a single server in a single datacenter. MongoDB Atlas is deployed to VMs on 2-3 AZs. You don't have close to the same reliability. (I'm also curious why their M40 instance costs $1000, when the Pricing Calculator ( https://www.mongodb.com/pricing )…

> You could cut your MongoDB costs by 100% by not using it ;)

Came here to say exactly this

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#174
What most of the commenters here are missing is the reality that not every system, function, or business needs the sort of uptime that AWS offers - and that's fine. It's something a lot of newer entrants into the technology field fail to grasp, because they've never had to actually deal with an outage before - or a time when the internet itself was ephemeral and temporary, available only as long as your connection remained active.

The number one thing people poo-pooing these "We saved $XXX by getting off public cloud" posts is that each business has different calculus for its risk tolerances, business needs, and opportunity costs. Once a function reaches some form of stability or homeostasis, then hosting it in the public cloud can become a net liability rather than a net asset. Being able to make those decisions impartially is what separates the genuinely good talent from those who conflate TC with wisdom.

Even when public cloud is the right decision, using managed services increasingly isn't. MongoDB Atlas is a managed service with a corresponding price tag to match. Running it in a VPS like Hetzner may shift some of the maintenance and support tasks onto your team, but let's be real - modern databases are designed to be bulletproof, and huge companies operated just fine with a single database instance on bare metal for decades, even with the odd downtime along the way. We ran a MongoDB CE database at a PriorCo on a single VM in a single datacenter for nearly a decade, and it underpinned a substantial chunk of our operations - operations we could do by hand, if needed, during downtime or outages (that never happened). We eventually moved it to AWS DocumentDB not out of cost-savings or necessity, but because a higher-up demanded we do so.

If anything, the visceral rebuke of anyone daring to move off public cloud feels very reminiscent of my own collegial douchebagginess in the 2000s, loudly mocking Linux stans and proclaiming closed source (Microsoft) would run the planet. Past-me was a douchebag then, and the same applies to the AWS-stans of today.

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#175

Earlier quoted context omitted.

Premature optimization. Not every single service needs or require 5 nines.

What does that mean, though? If I'm storing data on a NAS, and I keep backups on a tape, a simple hardware failure that causes zero downtime on S3 might take what, hours to recover? Days? If my database server dies and I need to boot a new one, how long will that take? If I'm on RDS, maybe five minutes. If it's bare metal and I need to install software and load my data into it, perhaps an hour or more. Being able to…

I know one company that strove for five sixes.

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#177
$2700/mo is about 1/3 of an engineers' salary (cost to the business of a mid-level engineer in the UK)...

But, there's the time to set all of this up (which admittedly is a one-time investment and would amortize).

And there's the risk of having made a mistake in your backups or recovery system (Will you exercise it? Will you continue to regularly exercise it?).

And they're a 3-person team... is it really worth your limited time/capacity to do this, rather than do something that's likely to attract $3k/mo of new business?

If the folks who wrote the blog see this, please share how much time (how many devs, how many weeks) this took to set up, and how the ongoing maintenance burden shapes up.

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#178
post #59

Earlier quoted context omitted.

> many people just chuck something on us-east-1 and go Even dropping something on a single EC2 node in us-east-1 (or at Google Cloud) is going to be more reliable over time than a single dedicated machine elsewhere. This is because they run with a layer that will e.g. live migrate your running apps in case of hardware failures. The failure modes of dedicated are quite different than those of the modern hyperscaler cl…

Hardware failures on server hardware at the scale of 1 machine are far less common than us-east-1 downtime The typical failure mode of AWS is much better. Half the internet is down, so you just point at that and wait for everything to come back, and your instances just keep running. If you have one server you have to do the troubleshooting and recovery work. But you need to run more than one machine to get fewer nine…

> Hardware failures on server hardware at the scale of 1 machine are far less common than us-east-1 downtime

A couple pieces of gentle pushback here:

- if you chose a hyperscaler, you should use their (often one-click) geographic redundancy & failover.

- All of the hyperscalers have more than one AZ. Specifically, there's no reason for any AWS customer to locate all/any* of their resources in us-east-1. (I actively recommend against this.)

* - Except for the small number of services only available in us-east-1, obviously.

Re: We cut our Mongo DB costs by 90% by moving to Hetzner

#180
post #68

Earlier quoted context omitted.

> we never had any issues, because we didn't depend on calling AWS APIs to continue operating. Things already running continue to run. I think it was just luck of the draw that the failure happened in this way and not some other way. Even if APIs falling over but EC2 instances remaining up is a slightly more likely failure mode, it means you can't run autoscaling, can't depend on spot instances which in an outage you…

> it means you can't run autoscaling, can't depend on spot instances which in an outage you can lose and can't replace Yes, this is part of designing for reliability. If you use spot or autoscaling, you can't assume you will have high availability in those components. They're optimizations, like a cache. A cache can disappear, and this can have a destabilizing effect on your architecture if you don't plan for it. Thi…

Sure, you can only use EC2, not use autoscaling or spot and instead just provision to your highest capacity needs, and not use any other AWS service that relies on dynamo as a dependency.

We still take some steps to mitigate control plane issues in what I consider a reasonable AWS setup (attempt to lock ASGs to prevent scale-down) but I place the control plane disappearing on the same level as the entire region going dark, and just run multi-region.

Post reply on HN