Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

311–320 of 407 posts

Re: Go ahead, self-host Postgres

#311

Post by dolmen »

nemothekid wrote: post #227
belorn wrote: post #219

Earlier quoted context omitted.

You can outsource everything, but outsourcing critical parts of the company may also put the existence of the company in the hand of a third-party. Is that an acceptable risk? Control and risk management cost money, be that by self hosting or contracts. At some point it is cheaper to buy the competence and make it part of the company rather than outsource it.

I think you and I simply disagree about your database being a core/critical part of your stack. I believe RDS is good enough for most people, and the only advantage you would have in self hosting is shaving 33% off your instance bill. I'd probably go a step further and argue that Neon/CockroachDB Serverless is good enough for most people.

Access control to your (customer's) data may also be a concern that rules out managed services like RDS.

Re: Go ahead, self-host Postgres

#312

Post by sixdonuts »

OccamsMirror wrote: post #78
laz wrote: post #38

Earlier quoted context omitted.

The worst SEV calls are the one where you twiddle your thumbs waiting for a support rep to drop a crumb of information about the provider outage. You wake up. It's not your fault. You're helpless to solve it.

Not when that provider is AWS and the outage is hitting news websites. You share the link to AWS being down and go back to sleep.

News is one thing, if the app/service down impacts revenue, safety or security you won't be getting any sleep AWS or not.

Re: Go ahead, self-host Postgres

#313

Post by foo42 »

gopher_space wrote: post #260
zbentley wrote: post #211

Earlier quoted context omitted.

No argument here, that’s a fair and thoughtful response, and you’re not wrong regarding the prejudice against self-hosting (and for what it’s worth I absolutely come from the era where that was the default approach, have done it extensively, like it, and still do it/recommend it when it makes sense). > “ geek code" thing we were doing a while back Not sure what you’re referring to. “Shibboleet”, perhaps? https://xkcd…

> The Geek Code, developed in 1993, is a series of letters and symbols used by self-described "geeks" to inform fellow geeks about their personality, appearance, interests, skills, and opinions. The idea is that everything that makes a geek individual can be encoded in a compact format which only other geeks can read. This is deemed to be efficient in some sufficiently geeky manner. https://en.wikipedia.org/wiki/Geek…

geek code is worthy of its own hn submission

Re: Go ahead, self-host Postgres

#314

Post by sixdonuts »

runako wrote: post #255
zbentley wrote: post #127

Earlier quoted context omitted.

I’ve worked for banks, multiple large and small US healthcare-related companies, and businesses that didn’t use their software when they were closed for the night. All of those places needed their backend systems to be up 24/7. The banks ran reports and cleared funds with nightly batches—hundreds of jobs a night for even small banking networks. The healthcare companies needed to receive claims and process patient upd…

You are right that a lot of systems at a lot of places need 24x7. Obviously. But there are also a not-insignificant number of important systems where nobody is on a pager, where there is no call rotation[1]. Computers are much more reliable than they were even 20 years ago. It is an Acceptable Business Choice to not have 24x7 monitoring for some subset of systems. Until very recently[2], Citibank took their public we…

Thousands of orgs have full stack OT/CI apps/services that must run 24/7 365 and are run fully on premise.

Re: Go ahead, self-host Postgres

#316

Post by npn »

RadiozRadioz wrote: post #236
groundzeros2015 wrote: post #230

Earlier quoted context omitted.

Engineers almost never consider any of those questions. And instead deploy the maximally expensive solution their boss will say ok to.

Bad, short-sighted engineers will do that. An engineer who is not acting solely in the best interests of the wider organisation is a bad one. I would not want to work with a colleague who was so detached from reality that they wouldn't consider all GP's suggested facets. Engineering includes soft/business constraints as well as technical ones.

I bet you also believe database is the single source of truth, right?

Re: Go ahead, self-host Postgres

#317

Post by drchaim »

I’ve been managing a 100+ GB PostgreSQL database for years. Each two years I upgrade the VPS for the size, and also the db and os version. The app is in the same VPS as the DB. A 2 hour window each two years is ok for the use case. No regrets.

Re: Go ahead, self-host Postgres

#318

Post by kunley »

petterroea wrote: post #66

I have ran (read: helped with infrastructure) a small production service using PSQL for 6 years, with up to hundreds of users per day. PSQL has been the problem exactly once, and it was because we ran out of disk space. Proper monitoring (duh) and a little VACUUM would have solved it. Later I ran a v2 of that service on k8s. The architecture also changed a lot, hosting many smaller servers sharing the same psql serve…

Psql (lowercase) is the name of the textual sql client for PostgreSQL. For a general abbreviation we rather use "Pg".

Re: Go ahead, self-host Postgres

#319

Post by lukaslalinsky »

I'm not a cloud-hosting fan, but comparing RDS to a single instance DB seems crazy to me. Even for a hobby project, I couldn't accept losing data since the last snapshot. If you are going to self-host PostgreSQL in production, make sure you have at least some knowledge how to setup streaming replication and have monitoring in place making sure the replication works. Ideally, use something like Patroni for automatic failover. I'm saying this a someone running fairly large self-hosted HA PostgreSQL databases in production.

Re: Go ahead, self-host Postgres

#320

Post by jillesvangurp »

There are a couple of things that are being glossed over:

Hardware failures and automated fail overs. That's a thing AWS and other managed hosting solutions do. Hardware will eventually fail of course. In AWS this would be a non event. It will fail over, a replacement spins up, etc. Same with upgrades, and other stuff.

Configuration complexity. The author casually outlines a lot of fairly complex design involving all sorts of configuration tweaks, load balancing, etc. That implies skills most teams don't have. I know enough to know that I have quite a bit of reading up to do if I ever were to decide to self host postgresql. Many people would make bad assumptions about things being fine out of the box because they are not experienced postgresql DBAs.

Vacations/holidays/sick days. Databases may go down when it's not convenient to you. To mitigate that, you need to have several colleagues that are equally qualified to fix things when they go down while you are away from keyboard. If you haven't covered that risk, you are taking a bit of risk. In a normal company, at least 3-4 people would be a good minimum. If you are just measuring your own time, you are not being honest or not being as diligent as you should be. Either it's a risk you are covering at a cost or a risk you are ignoring.

With managed hosting, covering all of that is what you pay for. You are right that there are still failure modes beyond that that need covering. But an honest assessment of the time you, and your team, put in for this adds up really quickly.

Whatever the reasons you are self hosting, cost is probably a poor one.

Post reply on HN