Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

291–300 of 407 posts

Re: Go ahead, self-host Postgres

#291

Post by Nextgrid »

strken wrote: post #273
Nextgrid wrote: post #176

Earlier quoted context omitted.

If they just paid half of the markup they currently pay for the cloud I'm sure they'll be swimming in qualified candidates.

Our AWS spend is something like $160/month. Want to come build bare metal database infrastructure for us for $3/day?

When you need to scale up and don't want that $160 to increase 10x to handle the additional load the numbers start making more sense: 3 month's worth of the projected increase upfront is around 4.3k, which is good money for a few days' work for the setup/migration and remains a good deal for you since you break even after 3 months and keep on pocketing the savings indefinitely from that point on.

Of course, my comment wasn't aimed at those who successfully keep their cloud bill in the low 3-figures, but the majority of companies with a 5-figure bill and multiple "infrastructure" people on payroll futzing around with YAML files. Even half the achieved savings should be enough incentive for those guys to learn something new.

Re: Go ahead, self-host Postgres

#292

Post by Seattle3503 »

danaris wrote: post #88
paulryanrogers wrote: post #19

Earlier quoted context omitted.

IMO Maria has fallen behind MySQL. I wouldn't chose it for anything my income depends on. (I do use Maria at home for legacy reasons, and have used MySQL and Pg professionally for years.)

> IMO Maria has fallen behind MySQL. I wouldn't chose it for anything my income depends on. Can you give any details on that? I switched to MariaDB back in the day for my personal projects because (so far as I could tell) it was being updated more regularly, and it was more fully open source. (I don't recall offhand at this point whether MySQL switched to a fully paid model, or just less-open.)

SKIP LOCKED was added in 10.6 (~2021), years after MySQL had it (~2017). My company was using MariaDB around the time and was trailing a version or two and it made implementing a queue very painful.

Re: Go ahead, self-host Postgres

#293

Post by bigstrat2003 »

_superposition_ wrote: post #263
bigstrat2003 wrote: post #256

Earlier quoted context omitted.

> Like everything, it is always cheaper to do it (it being cooking at home, cleaning your home, fixing your own car, etc) yourself (if you don't include the cost of your own time doing the service you normally pay someone else for). In a business context the "time is money" thing actually makes sense, because there's a reasonable likelihood that the business can put the time to a more profitable use in some other way…

Lol this made me laugh, there's a reasonable likelihood that time will be filled with meetings.

Heh, true. Although in fairness I said the business can repurpose the time to make money, not that they will. I'm splitting hairs, but it seems in keeping with the ethos here. ;)

Re: Go ahead, self-host Postgres

#294

Post by true_religion »

marcosdumay wrote: post #41
ijustlovemath wrote: post #8

Earlier quoted context omitted.

There's not much to recommend; just use the Postgres from your distribution's LTS repo. I like Debian for its rock solid stability.

The one problem with using your distro's Postgres is that your upgrade routine will be dictated by a 3rd party. And Postgres upgrades are not transparent. So you'll have a 1 or 2 hours task, every 6 to 18 months that you have only a small amount of control over when it happens. This is ok for a lot of people, and completely unthinkable for some other people.

Why would your distro dictate the upgrade routine? Unless the distro stops supporting an older version of Postgres, you can continue using it. Most companies I know of wouldn't dare do an upgrade of an existing production database for at least 5 years, and when it does happen... downtime is acceptable.

Re: Go ahead, self-host Postgres

#295

Post by westurner »

andersmurphy wrote: post #194
alemanek wrote: post #174

Earlier quoted context omitted.

As someone who has set this up while not being a DBA or sysadmin. Replication and backups really aren’t that difficult to setup properly with something like Postgres. You can also expose metrics around this to setup alerting if replication lag goes beyond a threshold you set or a backup didn’t complete. You do need to periodically test your backups but that is also good practice. I am not saying something like RDS do…

Even easier with sqlite thanks to litestream.

datasette and datasette-lite (WASM w/pyodide) are web UIs for SQLite with sqlite-utils.

For read only applications, it's possible to host datasette-lite and the SQLite database as static files on a redundant CDN. Datasette-lite + URL redirect API + litestream would probably work well, maybe with read-write; though also electric-sql has a sync engine (with optional partial replication) too, and there's PGlite (Postgres in WebAssembly)

Re: Go ahead, self-host Postgres

#296

Post by vbezhenar »

I don't feel like it's easy to self-host postgres.

Here are my gripes:

1. Backups are super-important. Losing production data just is not an option. Postgres offers pgdump which is not appropriate tool, so you should set up WAL archiving or something like that. This is complicated to do right.

2. Horizontal scalability with read replicas is hard to implement.

3. Tuning various postgres parameters is not a trivial task.

4. Upgrading major version is complicated.

5. You probably need to use something like pgbouncer.

6. Database usually is the most important piece of infrastructure. So it's especially painful when it fails.

I guess it's not that hard when you did it once and have all scripts and memory to look back. But otherwise it's hard. Clicking few buttons in hoster panel is much easier.

Re: Go ahead, self-host Postgres

#297

Post by true_religion »

notaseojh wrote: post #27
ijustlovemath wrote: post #8

Earlier quoted context omitted.

There's not much to recommend; just use the Postgres from your distribution's LTS repo. I like Debian for its rock solid stability.

"just use postgres from your distro" is *wildly* underselling the amount of work that it takes to go from apt install postgres to having a production ready setup (backups, replica, pooling, etc). Granted, if it's a tiny database just pg-dumping might be enough, but for many that isn't going to be enough.

If you're a 'startup', you'll never need any of that work until you make it big. 99% of startups do not make it even medium size.

If you're a small business, you almost never need replicas or pooling. Postgres is insanely capable on modern hardware, and is probably the fastest part of your application if your application is written in a slower dynamic language like Python.

I once worked with a company that scaled up to 30M revenue annually, and never once needed more than a single dedicated server for postgres.

Re: Go ahead, self-host Postgres

#298

Post by true_religion »

stickfigure wrote: post #278
runako wrote: post #255

Earlier quoted context omitted.

You are right that a lot of systems at a lot of places need 24x7. Obviously. But there are also a not-insignificant number of important systems where nobody is on a pager, where there is no call rotation[1]. Computers are much more reliable than they were even 20 years ago. It is an Acceptable Business Choice to not have 24x7 monitoring for some subset of systems. Until very recently[2], Citibank took their public we…

This lasts right up until an important customer can't access your services. Executives don't care about downtime until they have it, then they suddenly care a lot.

You can often have services available for VIPs, and be down for the public.

Unless there's a misconfiguration, usually apps are always visible internally to staff, so there's an existing methodology to follow to make them visible to VIPs.

But sometimes none of that is necessary. I've seen at a 1B market cap company, a failure case where the solution was manual execution by customer success reps while the computers were down. It was slower, but not many people complained that their reports took 10 minutes to arrive after being parsed by Eye Ball Mk 1s, instead of the 1 minute of wait time they were used to.

Re: Go ahead, self-host Postgres

#299

Post by nrhrjrjrjtntbt »

vbezhenar wrote: post #296

I don't feel like it's easy to self-host postgres. Here are my gripes: 1. Backups are super-important. Losing production data just is not an option. Postgres offers pgdump which is not appropriate tool, so you should set up WAL archiving or something like that. This is complicated to do right. 2. Horizontal scalability with read replicas is hard to implement. 3. Tuning various postgres parameters is not a trivial tas…

Scaling to a different instance size is also easy on AWS.

That said a self hosted DB on a dedicated Hetzner flies. It does things at the price that may save you time reworking your app to be more efficient on AWS for cost.

So swings and roundabouts.

Re: Go ahead, self-host Postgres

#300

Post by ajmurmann »

yunwal wrote: post #285
dilyevsky wrote: post #261

Earlier quoted context omitted.

This ignores the case when BigVendor is down for your account and your account only and support is mia, which is not that uncommon ime

It doesn’t ignore that case, it simply allows them to shift blame whereas the no name vendor does not.

"Nobody has ever been fired for buying IBM"
Post reply on HN