Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

241–250 of 407 posts

Re: Go ahead, self-host Postgres

#241

Post by jeremyjh »

dangoodmanUT wrote: post #32
ZeroConcerns wrote: post #11

So, yeah, I guess there's much confusion about what a 'managed database' actually is ? Because for me, the table stakes are: -Backups: the provider will push a full generic disaster-recovery backup of my database to an off-provider location at least daily, without the need for a maintenance window -Optimization: index maintenance and storage optimization are performed automatically and transparently -Multi-datacenter…

There should be no data loss window with a hosted database

That requires synchronous replication, which reduces availability and performance.

Re: Go ahead, self-host Postgres

#242

Post by SoftTalker »

awestroke wrote: post #146
satvikpendem wrote: post #138

Earlier quoted context omitted.

One thing I learned working in the industry, you have to check them when you're using AWS too.

Really? You're saying RDS backups can't be trusted?

No backup strategy can be blindly trusted. You must verify it, and also test that restores actually work.

Re: Go ahead, self-host Postgres

#243

Post by [deleted] »

belorn wrote: post #219
nemothekid wrote: post #195

Earlier quoted context omitted.

Unironically - I agree. You should be outsourcing things that aren't your core competency. I think many people on this forum have a certain pride about doing this manually, but to me it wouldn't make sense in any other context. Could you imagine accountants arguing that you shouldn't use a service like Paychex or Gusto and just run payroll manually? After all it's cheaper! Just spend a week tracking taxes, benefits a…

You can outsource everything, but outsourcing critical parts of the company may also put the existence of the company in the hand of a third-party. Is that an acceptable risk? Control and risk management cost money, be that by self hosting or contracts. At some point it is cheaper to buy the competence and make it part of the company rather than outsource it.

[deleted]

Re: Go ahead, self-host Postgres

#244

Post by conradfr »

I've been self hosting Postgresql for 12+ years at this point. Directly on bare metal then and now in a container with CapRover.

I have a cron sh script to backup to S3 (used to be ftp).

It's not "business grade" but it has also actually NEVER failed. Well once, but I think it was more the container or a swarm thing. I just destroyed and recreated it and it picked up the same volume fine.

The biggest pain point is upgrading as Postgresql can't upgrade the data without the previous version installed or something. It's VERY annoying.

Re: Go ahead, self-host Postgres

#245

Post by jread »

cobolcomesback wrote: post #233
jread wrote: post #199

Earlier quoted context omitted.

This was true for RDS serverless v1 which scaled to 0 but is no longer offered. V2 requires a minimum 0.5 ACU hourly commit ($40+ /mo).

V2 scales to zero as of last year. https://aws.amazon.com/blogs/database/introducing-scaling-to... It only scales down after a period of inactivity though - it’s not pay-per-request like other serverless offerings. DSQL looks to be more cost effective for small projects if you can deal with the deviations from Postgres.

Ah, good to know, I hadn't seen that V2 update. Looks like a min 5m inactivity to auto-pause (i.e., scale to 0), and any connection attempt (valid or not) resumes the DB.

Re: Go ahead, self-host Postgres

#246

Post by scott_w »

pinkgolem wrote: post #192
scott_w wrote: post #186

Earlier quoted context omitted.

A Django+Celery app behind Nginx back in the day. Most maintenance would be discovering a new failure mode: - certificates not being renewed in time - Celery eating up all RAM and having to be recycled - RabbitMQ getting blocked requiring a forced restart - random issues with Postgres that usually required a hard restart of PG (running low on RAM maybe?) - configs having issues - running out of inodes - DNS not updat…

I mean, going back a decade might be part of the reason? Configs having issues is like number 1 reason i like the setup so much.. I can configure everything on my local machine and test here, and then just deploy it to a server the same way. I do not have to build a local setup, and then a remote one

Er… what? Even in today’s world with Docker, you have differences between dev and prod. For a start, one is accessed via the internet and requires TLS configs to work correctly. The other is accessed via localhost.

Re: Go ahead, self-host Postgres

#247

Post by computerfan494 »

abrookewood wrote: post #239
franckpachot wrote: post #92

Earlier quoted context omitted.

Beyond the hype, the PostgreSQL community is aware of the lack of "batteries-included" HA. This discussion on the idea of a Built-in Raft replication mentions MongoDB as: >> "God Send". Everything just worked. Replication was as reliable as one could imagine. It outlives several hardware incidents without manual intervention. It allowed cluster maintenance (software and hardware upgrades) without application downtime…

"I really dream PostgreSQL will be as reliable as MongoDB" ... someone needs to go and read up on Mongo's history! Sure, the PostrgreSQL HA story isn't what we all want it to be, but the reliability is exceptional.

Postgres violated serializability on a single node for a considerable amount of time [1] and used fsync incorrectly for 20 years [2]. I personally witnessed lost data on Postgres because of the fsync issue.

Database engineering is very hard. MongoDB has had both poor defaults as well as bugs in the past. It will certainly have durability bugs in the future, just like Postgres and all other serious databases. I'm not sure that Postgres' durability stacks up especially well with modern MongoDB.

[1] https://jepsen.io/analyses/postgresql-12.3

[2] https://archive.fosdem.org/2019/schedule/event/postgresql_fs...

Re: Go ahead, self-host Postgres

#248

Post by SamDc73 »

satvikpendem wrote: post #106
ijustlovemath wrote: post #5

And if you want a supabase-like functionality, I'm a huge fan of PostgREST (which is actually how supabase works/worked under the hood). Make a view for your application and boom, you have a GET only REST API. Add a plpgsql function, and now you can POST. It uses JWT for auth, but usually I have application on the same VLAN as DB so it's not as rife for abuse.

You can self host Supabase too.

Last time I checked, it was a pain in the ass to self-host it

Re: Go ahead, self-host Postgres

#249

Post by erulabs »

Nextgrid wrote: post #42
molf wrote: post #36

> I'd argue self-hosting is the right choice for basically everyone, with the few exceptions at both ends of the extreme: > If you're just starting out in software & want to get something working quickly with vibe coding, it's easier to treat Postgres as just another remote API that you can call from your single deployed app > If you're a really big company and are reaching the scale where you need trained database e…

> employing engineers to manage self-hosted databases is more cost effective than outsourcing Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. PaaS platforms (Heroku, Render, Railway) can legitimately be operated by your average dev and not have to hire a dedicated person; those cost even mo…

Well sure you still have 2 or 3 infra people but now you don’t need 15. Comparing to modern Hetzner is also not fair to “cloud” in the sense that click-and-get-server didn’t exist until cloud providers popped up. That was initially the whole point. If bare metal behind an API existed in 2009 the whole industry would look very different. Contingencies Rule Everything Around Me.

Re: Go ahead, self-host Postgres

#250

Post by api »

tjwebbnorfolk wrote: post #226
madeofpalk wrote: post #182

Earlier quoted context omitted.

> but it allows me to say, "we'll have to wait until they've sorted this out, Ikea and Disney are down, too." From my experience your client’s clients don’t care about this when they’re still otherwise up.

Yes but the fact that it's "not their fault" keeps the person from getting fired. Don't underestimate the power of CYA

This is a major reason the cloud commands such a premium. It’s a way to make down time someone else’s problem.

The other factor is eliminating the “one guy who knows X” problem in IT. What happens if that person leaves or you have to let them go? But with managed infrastructure there’s a pool of people who know how to write terraform or click buttons and manage it and those are more interchangeable than someone’s DIY deployment. Worst case the cloud provider might sell you premium support and help. Might be expensive but you’re not down.

Lastly, there’s been an exodus of talent from IT. The problem is that anyone really good can become a coder and make more. So finding IT people at a reasonable cost who know how to really troubleshoot and root cause stuff and engineer good systems is very hard. The good ones command more of a programmer salary which makes the gap with cloud costs much smaller. Might as well just go managed cloud.

Post reply on HN