Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

211–220 of 407 posts

Re: Go ahead, self-host Postgres

#211

Post by zbentley »

gopher_space wrote: post #208
zbentley wrote: post #135

Earlier quoted context omitted.

That’s pretty reductive. By that logic the opposite extreme is just as true: if using managed services is just as bad as outsourcing everything else, then a business shouldn’t rent real estate either—every business should build and own their own facility. They should also never contract out janitorial work, nor should they retain outside law firms—they should hire and staff those departments internally, every time, n…

I'll be reductive in conversations like this just to help push the pendulum back a little. The prevailing attitude seems (to me) like people find self-hosting mystical and occult, yet there's never been a better time to do it. > But it’s very case-by-case. There’s no general rule like “always prefer self hosting” or “always rent real estate, never buy” that applies broadly enough to be useful. I don't know if anyone…

No argument here, that’s a fair and thoughtful response, and you’re not wrong regarding the prejudice against self-hosting (and for what it’s worth I absolutely come from the era where that was the default approach, have done it extensively, like it, and still do it/recommend it when it makes sense).

> “ geek code" thing we were doing a while back

Not sure what you’re referring to. “Shibboleet”, perhaps? https://xkcd.com/806/

Re: Go ahead, self-host Postgres

#212

Post by devin »

What irks me about so many comments in this thread is that they often totally ignore questions of scale, the shape of your workloads, staffing concerns, time constraints, stage of your business, whether you require extensions, etc.

There is a whole raft of reasons why you might be a candidate for self-hosting, and a whole raft of reasons why not. This article is deeply reductive, and so are many of the comments.

Re: Go ahead, self-host Postgres

#213

Post by roncesvalles »

I'd argue forget about Postgres completely. If you can shell out $90/month, the only database you should use is GCP Spanner (yes, this also means forget about any mega cloud other than GCP unless you're fine paying ingress and egress).

And for small projects, SQLite, rqlite, or etcd.

My logic is either the project is important enough that data durability matters to you and sees enough scale that loss of data durability would be a major pain in the ass to fix, or the project is not very big and you can tolerate some lost committed transactions.

A consensus-replication-less non-embedded database has no place in 2025.

This is assuming you have relational needs. For non-relational just use the native NoSQL in your cloud, e.g. DynamoDB in AWS.

Re: Go ahead, self-host Postgres

#214

Post by solarengineer »

ErroneousBosh wrote: post #58
solarengineer wrote: post #35

Earlier quoted context omitted.

Have you given thought to why you prototype with SQLite? I have switched to using postgres even for prototyping once I prepared some shell scripts for various setup. With hibernate (java) or knex (Javascript/NodeJS) and with unit tests (Test Driven Development approach) for code, I feel I have reduced the friction of using postgres from the beginning.

Because when I get tired of reconstructing the contents of the database between my various dev machines (at home, at work, on a remote server, on my laptop) I can just scp the sqlite db across. Because it's "low effort" to just fire it into sqlite and if I have to do ridiculous things to the schema as I footer around working out exactly what I want the database to do. I don't want to use nodejs if I can possibly avoi…

I mentioned Hibernate and knex as examples of DB schema version control tools.

Incidentally, you can rsync postgres dumps as well. That's what I do when testing and when sharing test data with team mates. At times, I decide to pgload the database dump into a different target system.

My reason for sharing: I accepted that I was being lethargic about using postgres, so I just automated certain things as I went along.

Re: Go ahead, self-host Postgres

#215

Post by andersmurphy »

Nextgrid wrote: post #83
convolvatron wrote: post #40

Earlier quoted context omitted.

its not. I've been in a few shops that use RDS because they think their time is better spend doing other things. except now they are stuck trying to maintain and debug Postgres without having the same visibility and agency that they would if they hosted it themselves. situation isn't at all clear.

One thing unaccounted for if you've only ever used cloud-hosted DBs is just how slow they are compared to a modern server with NVME storage. This leads the developers to do all kinds of workarounds and reach for more cloud services (and then integrating them and - often poorly - ensuring consistency across them) because the cloud hosted DB is not able to handle the load. On bare-metal, you can go a very long way with…

100% this directly connected nvme is a massive win. Often several orders of magnitude.

You can take it even further in some context if you use sqlite.

I think one of the craziest ideas of the cloud decade was to move storage away from compute. It's even worse with things like AWS lambda or vercel.

Now vercel et al are charging you extra to have your data next to your compute. We're basically back to VMs at 100-1000x the cost.

Re: Go ahead, self-host Postgres

#216

Post by andersmurphy »

marcosdumay wrote: post #26
ZeroConcerns wrote: post #11

So, yeah, I guess there's much confusion about what a 'managed database' actually is ? Because for me, the table stakes are: -Backups: the provider will push a full generic disaster-recovery backup of my database to an off-provider location at least daily, without the need for a maintenance window -Optimization: index maintenance and storage optimization are performed automatically and transparently -Multi-datacenter…

IMO, the reason to self-host your database is latency. Yes, I'd say backups and analysis are table stakes for hiring it, and multi-datacenter failover is a relevant nice to have. But the reason to do it yourself is because it's literally impossible to get anything as good as you can build in somebody's else computer.

Yup, often orders of magnitude better.

Re: Go ahead, self-host Postgres

#218

Post by andersmurphy »

dangoodmanUT wrote: post #32
ZeroConcerns wrote: post #11

So, yeah, I guess there's much confusion about what a 'managed database' actually is ? Because for me, the table stakes are: -Backups: the provider will push a full generic disaster-recovery backup of my database to an off-provider location at least daily, without the need for a maintenance window -Optimization: index maintenance and storage optimization are performed automatically and transparently -Multi-datacenter…

There should be no data loss window with a hosted database

Feom what I remember if AWS loses your data they are basically give you some credits and that's it.

Re: Go ahead, self-host Postgres

#219

Post by belorn »

nemothekid wrote: post #195
jbverschoor wrote: post #129

Earlier quoted context omitted.

Yea I agree.. better outsource product development, management, and everything else too by that narrative

Unironically - I agree. You should be outsourcing things that aren't your core competency. I think many people on this forum have a certain pride about doing this manually, but to me it wouldn't make sense in any other context. Could you imagine accountants arguing that you shouldn't use a service like Paychex or Gusto and just run payroll manually? After all it's cheaper! Just spend a week tracking taxes, benefits a…

You can outsource everything, but outsourcing critical parts of the company may also put the existence of the company in the hand of a third-party. Is that an acceptable risk?

Control and risk management cost money, be that by self hosting or contracts. At some point it is cheaper to buy the competence and make it part of the company rather than outsource it.

Re: Go ahead, self-host Postgres

#220

Post by reilly3000 »

petterroea wrote: post #66

I have ran (read: helped with infrastructure) a small production service using PSQL for 6 years, with up to hundreds of users per day. PSQL has been the problem exactly once, and it was because we ran out of disk space. Proper monitoring (duh) and a little VACUUM would have solved it. Later I ran a v2 of that service on k8s. The architecture also changed a lot, hosting many smaller servers sharing the same psql serve…

But it’s 1500 pages long!
Post reply on HN