Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

331–340 of 407 posts

Re: Go ahead, self-host Postgres

#331

Post by growse »

01HNNWZ0MV43FF wrote: post #254
api wrote: post #250

Earlier quoted context omitted.

This is a major reason the cloud commands such a premium. It’s a way to make down time someone else’s problem. The other factor is eliminating the “one guy who knows X” problem in IT. What happens if that person leaves or you have to let them go? But with managed infrastructure there’s a pool of people who know how to write terraform or click buttons and manage it and those are more interchangeable than someone’s DIY…

That is called "bus factor" or "lottery factor". If the one IT guy gets hit by a bus or wins the lottery and quits, what happens? You want a bus factor of two or more - Two people would have to get hit by a bus for the company to have a big problem

There's a bus factor equivalent with the cloud, too. The power to severely disrupt your service (either accidentally, or on purpose) rests with a single org (and often, a single compliance department within that org).

Ironically, this becomes more of a concern the larger the supplier. AWS can live with firing any one of their customers - a smaller outfit probably couldn't.

Re: Go ahead, self-host Postgres

#332

Post by lucideer »

molf wrote: post #36

> I'd argue self-hosting is the right choice for basically everyone, with the few exceptions at both ends of the extreme: > If you're just starting out in software & want to get something working quickly with vibe coding, it's easier to treat Postgres as just another remote API that you can call from your single deployed app > If you're a really big company and are reaching the scale where you need trained database e…

> at a company that has reached the scale where employing engineers to manage self-hosted databases is more cost effective than outsourcing.

This is the crux of one of the most common fallacies in software engineering decision making today. I've participated in a bunch of architecture / vendor evaluations that concluded managed services are more cost effective almost purely because they underestimated (or even discarded entirely) the internal engineering cost of vendor management. Black box debugging is one of the most time costuming engineering pursuits, & even when it's something widely documented & well supported like RDS, it's only really tuned for the lowest common denominator - the complexities of tuning someone else's system at scale can really add up to only marginally less effort than self-hosting (if there's any difference at all).

But most importantly - even if it's significantly less effort than self-hosting, it's never effectively costed when evaluating trade-offs - that's what leads to this persistent myth about the engineering cost of self-hosting. "Managing" managed services is a non-zero cost.

Add to that the ultimate trade-off of accountability vs availability (internal engineers care less about availability when it's out of there hands - but it's still a loss to your product either way).

Re: Go ahead, self-host Postgres

#333

Post by danparsonson »

banditelol wrote: post #330

One of the things that made me think twice for self hosting postgres is securing the OS I host PG on. Any recommendation where to start for that?

Can you get away without exposing it to the internet? Firewall it off altogether, or just open the address of a specific machine that needs access to it?

Re: Go ahead, self-host Postgres

#334

Post by barnabee »

molf wrote: post #49
Nextgrid wrote: post #42

Earlier quoted context omitted.

> employing engineers to manage self-hosted databases is more cost effective than outsourcing Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. PaaS platforms (Heroku, Render, Railway) can legitimately be operated by your average dev and not have to hire a dedicated person; those cost even mo…

> Every company out there is using the cloud and yet still employs infrastructure engineers Every company beyond a particular size surely? For many small and medium sized companies hiring an infrastructure team makes just as little sense as hiring kitchen staff to make lunch.

It depends very much what the company is doing.

At my last two places it very quickly got to the point where the technical complexity of deployments, managing environments, dealing with large piles of data, etc. meant that we needed to hire someone to deal with it all.

They actually preferred managing VMs and self hosting in many cases (we kept the cloud web hosting for features like deploy previews, but that’s about it) to dealing with proprietary cloud tooling and APIs. Saved a ton of money, too.

On the other hand, the place before that was simple enough to build and deploy using cloud solutions without hiring someone dedicated (up to at least some pretty substantial scale that we didn’t hit).

Re: Go ahead, self-host Postgres

#335

Post by PunchyHamster »

Cooking the RDS equivalent is reasonable amount of work, and pretty big amount of knowledge (easy to make failover solution have lower uptime than "just a single VM" if you don't get everything right)

... but you can do a lot with just "a single VM and robust backup". PostgreSQL restore is pretty fast, and if you automated deployment you can start with it in minutes, so if your service can survive 30 minutes of downtime once every 3 years while the DB reloads, "downgrading" to "a single cloud VM" or "a single VM on your own hardware" might not be a big deal.

Re: Go ahead, self-host Postgres

#336

Post by PunchyHamster »

WJW wrote: post #325
zelphirkalt wrote: post #323

Earlier quoted context omitted.

So in the end it's not better for the users at all, it's just for non-technical people to shift blame. Great "business reasoning".

Nobody in this thread ever claimed it was better for the users. It's better for the people involved in the decision.

It's not tho, they have workers that they pay not making money, all while footing bigger bill for the "pleasure"

Re: Go ahead, self-host Postgres

#337

Post by PunchyHamster »

paulryanrogers wrote: post #160
mattmanser wrote: post #136

Earlier quoted context omitted.

Over 20 year I've had lots of clients on self-hosted, even self-hosting SQL on the same VM as the webserver as you used to in the long distant past for low-usage web apps. I have never, ever, ever had a SQL box go down. I've had a web server go down once. I had someone who probably shouldn't have had access to a server accidentally turn one off once. The only major outage I've had (2/3 hours) was when the box was als…

Disks go bad. RAID is nontrivial to set up. Hetzner had a big DC outage that lead to data loss. Off site backups or replication would help, though not always trivial to fail over.

They, do, it isn't, cloud providers also go bad.

> Off site backups or replication would help, though not always trivial to fail over.

You want those regardless of where you host

Re: Go ahead, self-host Postgres

#338

Post by forinti »

heipei wrote: post #15

I still don't get how folks can hype Postgres with every second post on HN, yet there is no simple batteries-included way to run a HA Postgres cluster with automatic failover like you can do with MongoDB. I'm genuinely curious how people deal with this in production when they're self-hosting.

I love Postgresql simply because it never gives me any trouble. I've been running it for decades without trouble.

OTOH, Oracle takes most of my time with endless issues, bugs, unexpected feature modifications, even on OCI!

Re: Go ahead, self-host Postgres

#339

Post by PunchyHamster »

bossyTeacher wrote: post #126
mittermayr wrote: post #116

Self-hosting is more a question of responsibility I'd say. I am running a couple of SaaS products and self-host at much better performance at a fraction of the cost of running this on AWS. It's amazing and it works perfectly fine. For client projects, however, I always try and sell them on paying the AWS fees, simply because it shifts the responsibility of the hardware being "up" to someone else. It does not inherent…

> Self-hosting is more a question of responsibility I'd say. I am running a couple of SaaS products and self-host at much better performance at a fraction of the cost of running this on AWS It is. You need to answer the question: what are the consecuences of your service being down for lets say 4 hours or some security patch isn't properly applied or you have not followed the best practices in terms of security? Many…

You can pay someone else to manage your hardware stack, there are literal companies that will just keep it running, while you just deploy your apps on that.

> It is. You need to answer the question: what are the consecuences of your service being down for lets say 4 hours or some security patch isn't properly applied or you have not followed the best practices in terms of security?

There is one advantage self hosted setup has here, if you set up VPN, only your employees have access, and you can have server not accessible from the internet. So even in case of zero day that WILL make SaaS company leak your data, you can be safe(r) with self-hosted solution.

> Your time is money though. You are saving money but giving up time.

The investment compounds. Setting up infra to run a single container for some app takes time and there is good chance it won't pay back for itself.

But 2nd service ? Cheaper. 5th ? At that point you probably had it automated enough that it's just pointing it at docker container and tweaking few settings.

> Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for).

It's cheaper if you include your own time. You pay a technical person at your company to do it. Saas company does that, then pays sales and PR person to sell it, then pays income tax to it, then it also needs to "pay" investors.

Yeah making a service for 4 people in company can be more work than just paying $10/mo to SaaS company. But 20 ? 50 ? 100 ? It quickly gets to point where self hosting (whether actually "self" or by using dedicated servers, or by using cloud) actually pays off

Re: Go ahead, self-host Postgres

#340

Post by jgalt212 »

Does anyone offer a managed database service where the database and your application server live on the same box? Until, I can get such latency advantages of such a set-up, we've found latency just too high to go with a managed solution. We are already spending too much batching or vectorizing database reads.
Post reply on HN