Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

61–70 of 407 posts

Re: Go ahead, self-host Postgres

#61

Post by cardanome »

molf wrote: post #36

> I'd argue self-hosting is the right choice for basically everyone, with the few exceptions at both ends of the extreme: > If you're just starting out in software & want to get something working quickly with vibe coding, it's easier to treat Postgres as just another remote API that you can call from your single deployed app > If you're a really big company and are reaching the scale where you need trained database e…

You are missing that most services don't have high availability needs and don't need to scale.

Most projects I have worked on in my career have never seen more than a hundred concurrent users. If something goes down on Saturday, I am going to fix it on Monday.

I have worked on internal tools were I just added a postgres DB to the docker setup and that was it. 5 Minute of work and no issues at all. Sure if you have something customer facing, you need to do a bit more and setup a good backup strategy but that really isn't magic.

Re: Go ahead, self-host Postgres

#62

Post by aranelsurion »

Nextgrid wrote: post #42
molf wrote: post #36

> I'd argue self-hosting is the right choice for basically everyone, with the few exceptions at both ends of the extreme: > If you're just starting out in software & want to get something working quickly with vibe coding, it's easier to treat Postgres as just another remote API that you can call from your single deployed app > If you're a really big company and are reaching the scale where you need trained database e…

> employing engineers to manage self-hosted databases is more cost effective than outsourcing Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. PaaS platforms (Heroku, Render, Railway) can legitimately be operated by your average dev and not have to hire a dedicated person; those cost even mo…

> still employs infrastructure engineers

> The "cloud" reducing staff costs

Both can be true at the same time.

Also:

> Otherwise you're waking up at 3am no matter what.

Do you account for frequency and variety of wakeups here?

Re: Go ahead, self-host Postgres

#63

Post by spwa4 »

molf wrote: post #49
Nextgrid wrote: post #42

Earlier quoted context omitted.

> employing engineers to manage self-hosted databases is more cost effective than outsourcing Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. PaaS platforms (Heroku, Render, Railway) can legitimately be operated by your average dev and not have to hire a dedicated person; those cost even mo…

> Every company out there is using the cloud and yet still employs infrastructure engineers Every company beyond a particular size surely? For many small and medium sized companies hiring an infrastructure team makes just as little sense as hiring kitchen staff to make lunch.

For small companies things like vercel, supabase, firebase, ... wipe the floor with Amazon RDS.

For medium sized companies you need "devops engineers". And in all honesty, more than you'd need sysadmins for the same deployment.

For large companies, they split up AWS responsibilities into entire departments of teams (for example, all clouds have math auth so damn difficult most large companies have -not 1- but multiple departments just dealing with authorization, before you so much as start your first app)

Re: Go ahead, self-host Postgres

#64

Post by citizenpaul »

orev wrote: post #46
donatj wrote: post #28

The author brings up the point, but I have always found surprising how much more expensive managed databases are than a comparable VPS. I would expect a little bit more as a cost of the convenience, but in my experience it's generally multiple times the expense. It's wild. This has kept me away from managed databases in all but my largest projects.

Once they convince you that you can’t do it yourself, you end up relying on them, but didn’t develop the skills you would need to migrate to another provider when they start raising prices. And they keep raising prices because by then you have no choice.

I have not tested this in real life yet but it seems like all the argument about vendor lock in can be solved, if you bite the bullet and learn basic Kubernetes administration. Kubernetes is FOSS and there are countless Kubernetes as a service providers.

I know there are other issues with Kubernetes but at least its transferable knowledge.

Re: Go ahead, self-host Postgres

#65

Post by zbentley »

arcbyte wrote: post #30
thedougd wrote: post #16

Earlier quoted context omitted.

It's even worse when you start finding you're staffing specialized skills. You have the Postgres person, and they're not quite busy enough, but nobody else wants to do what they do. But then you have an issue while they're on vacation, and that's a problem. Now I have a critical service but with a bus factor problem. So now I staff two people who are now not very busy at all. One is a bit ambitious and is tired of be…

Slack is a necessary component in well functioning systems.

And rental/SaaS models often provide an extremely cost effective alternative to needing to have a lot of slack.

Corollary: rental/SaaS models provide that property in large part because their providers have lots of slack.

Re: Go ahead, self-host Postgres

#66

Post by petterroea »

I have ran (read: helped with infrastructure) a small production service using PSQL for 6 years, with up to hundreds of users per day. PSQL has been the problem exactly once, and it was because we ran out of disk space. Proper monitoring (duh) and a little VACUUM would have solved it.

Later I ran a v2 of that service on k8s. The architecture also changed a lot, hosting many smaller servers sharing the same psql server(Not really microservice-related, think more "collective of smaller services ran by different people"). I have hit some issues relating to maxing out the max connections, but that's about it.

This is something I do on my free time so SLA isn't an issue, meaning I've had the ability to learn the ropes of running PSQL without many bad consequences. I'm really happy I have had this opportunity.

My conclusion is that running PSQL is totally fine if you just set up proper monitoring. If you are an engineer that works with infrastructure, even just because nobody else can/wants to, hosting PSQL is probably fine for you. Just RTFM.

Re: Go ahead, self-host Postgres

#67

Post by cromulent »

Without disagreeing:

Sometimes it is nice to simplify the conversation with non-tech management. Oh, you want HA / DR / etc? We click a button and you get it (multi-AZ). Clicking the button doubles your DB costs from x to y. Please choose.

Then you have one less repeating conversation and someone to blame.

Re: Go ahead, self-host Postgres

#68

Post by Nextgrid »

matthewmacleod wrote: post #48
Nextgrid wrote: post #42

Earlier quoted context omitted.

> employing engineers to manage self-hosted databases is more cost effective than outsourcing Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. PaaS platforms (Heroku, Render, Railway) can legitimately be operated by your average dev and not have to hire a dedicated person; those cost even mo…

I don’t think it’s a lie, it’s just perhaps overstated. The number of staff needed to manage a cloud infrastructure is definitely lower than that required to manage the equivalent self-hosted infrastructure. Whether or not you need that equivalence is an orthogonal question.

> The number of staff needed to manage a cloud infrastructure is definitely lower than that required to manage the equivalent self-hosted infrastructure.

There's probably a sweet spot where that is true, but because cloud providers offer more complexity (self-inflicted problems) and use PR to encourage you to use them ("best practices" and so on) in all the cloud-hosted shops I've been in a decade of experience I've always seen multiple full-time infra people being busy with... something?

There was always something to do, whether to keep up with cloud provider changes/deprecations, implementing the latest "best practice", debugging distributed systems failures or self-inflicted problems and so on. I'm sure career/resume polishing incentives are at play here too - the employee wants the system to require their input otherwise their job is no longer needed.

Maybe in a perfect world you can indeed use cloud-hosted services to reduce/eliminate dedicated staff, but in practice I've never seen anything but solo founders actually achieve that.

Re: Go ahead, self-host Postgres

#69

Post by freedomben »

matthewmacleod wrote: post #48
Nextgrid wrote: post #42

Earlier quoted context omitted.

> employing engineers to manage self-hosted databases is more cost effective than outsourcing Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. PaaS platforms (Heroku, Render, Railway) can legitimately be operated by your average dev and not have to hire a dedicated person; those cost even mo…

I don’t think it’s a lie, it’s just perhaps overstated. The number of staff needed to manage a cloud infrastructure is definitely lower than that required to manage the equivalent self-hosted infrastructure. Whether or not you need that equivalence is an orthogonal question.

Exactly. Companies with cloud infra often still have to hire infra people or even an infra team, but that team will be smaller than if they were self-hosting everything, in some cases radically smaller.

I love self-hosting stuff and even have a bias towards it, but the cost/time tradeoff is more complex than most people think.

Re: Go ahead, self-host Postgres

#70

Post by aranelsurion »

zbentley wrote: post #54
orev wrote: post #46

Earlier quoted context omitted.

Once they convince you that you can’t do it yourself, you end up relying on them, but didn’t develop the skills you would need to migrate to another provider when they start raising prices. And they keep raising prices because by then you have no choice.

There is plenty of provider markup, to be sure. But it is also very much not a given that the hosted version of a database is running software/configs that are equivalent to what you could do yourself. Many hosted databases are extremely different behind the scenes when it comes to durability, monitoring, failover, storage provisioning, compute provisioning, and more. Just because it acts like a connection hanging of…

> Just because it acts like a connection hanging off

If anything that’s a feature for ease of use and compatibility.

Post reply on HN