Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

141–150 of 407 posts

Re: Go ahead, self-host Postgres

#141

Post by [deleted] »

dvtkrlbs wrote: post #97
notaseojh wrote: post #27

Earlier quoted context omitted.

"just use postgres from your distro" is *wildly* underselling the amount of work that it takes to go from apt install postgres to having a production ready setup (backups, replica, pooling, etc). Granted, if it's a tiny database just pg-dumping might be enough, but for many that isn't going to be enough.

I don't think any of these would take more than a week to setup. Assuming you create a nice runbook with every step it would not be horrible to maintain as well. Barman for backups and unless you need multi-master you can use the builtin publication and subscription. Though with scale things can complicated really fast but most of the time you won't that much traffic to have something complicated.

[deleted]

Re: Go ahead, self-host Postgres

#142

Post by scott_w »

kijin wrote: post #117
scott_w wrote: post #80

Earlier quoted context omitted.

> Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. This doesn’t make sense as an argument. The reason the cloud is more complex is because that complexity is available. Under a certain size, a large number of cloud products simply can’t be managed in-house (and certainly not altogether). Als…

Yes, mix-and-match is the way to go, depending on what kind of skills are available in your team. I wouldn't touch a mail server with a 10-foot pole, but I'll happily self-manage certain daemons that I'm comfortable with. Just be careful not to accept more complexity just because it is available, which is what the AWS evangelists often try to sell. After all, we should always make an informed decision when adding a n…

Of course AWS are trying to sell you everything. It’s still on you and your team to understand your product and infrastructure and decide what makes sense for you.

Re: Go ahead, self-host Postgres

#143

Post by wredcoll »

Nextgrid wrote: post #77
aranelsurion wrote: post #62

Earlier quoted context omitted.

> still employs infrastructure engineers > The "cloud" reducing staff costs Both can be true at the same time. Also: > Otherwise you're waking up at 3am no matter what. Do you account for frequency and variety of wakeups here?

> Do you account for frequency and variety of wakeups here? Yes. In my career I've dealt with way more failures due to unnecessary distributed systems (that could have been one big bare-metal box) rather than hardware failures. You can never eliminate wake-ups, but I find bare-metal systems to have much less moving parts means you eliminate a whole bunch of failure scenarios so you're only left with actual hardware f…

If this isn't the truth. I just spent several weeks, on and off, debugging a remote hosted build system tool thingy because it was in turn made of at least 50 different microservice type systems and it was breaking in the middle of two of them.

There was, I have to admit, a log message that explained the problem... once I could find the specific log message and understand the 45 steps in the chain that got to that spot.

Re: Go ahead, self-host Postgres

#144

Post by zbentley »

LunaSea wrote: post #130
zbentley wrote: post #105

Earlier quoted context omitted.

Yep! I was mostly replying to TFA’s claim that AWS RDS is > Standard Postgres compiled with some AWS-specific monitoring hooks … and other operational tools deployed alongside it. That’s not always true: RDS classic may be those things, but RDS Aurora/Serverless is anything but. As to whether > self-hosted PostgreSQL on a bare metal server with NVMe SSDs will much faster than what RDS is capable of That’s often but n…

> Also, pedantically, you can run RDS on bare metal with local NVMEs. Only if you like your data to evaporate when the server stops. I'm relatively sure that the processing power and memory you can buy on OVH / Hetzner / co. is larger and cheaper even if you take into account peaks in your usage patterns.

> Only if you like your data to evaporate when the server stops.

(Edited to remove glib and vague rejoinder, sorry) Then hibernate/reboot it instead of stopping it? Alternatively, that’s what backup-to S3, periodic snapshot-to-EBS, clustering, or running an EBS-persisted zero-query-volume tiny replica are for.

> the processing power and memory you can buy on OVH / Hetzner / co. is larger and cheaper

Cheaper? Yeah, generally. But larger/more performant? Not always—it’s not about peaks/autoscaling, it’s about the (large) minority of workloads that will work better on RDS/Aurora/Serverless: auto-scale-out makes the reports run on time regardless of cost; bulk data loads are available on replicas a lot sooner on Aurora because the storage is the replication system, not the WAL; and so on—if you add up all the situations where the hosted RDBMS systems trump self hosted, you get an amount that’s not “hosted is always better/worth it”, but it’s not “hosted is just ops time savings and is otherwise just slower/more expensive” either. And that’s before you add reliability into the conversation.

Re: Go ahead, self-host Postgres

#145

Post by cosmosgenius »

adenta wrote: post #51

I wish this article would have went more in-depth on how they're setting up backups. The great thing about sequel light is lightstream makes backup and restore something you don't really have to think about

for postgres specifically pgbackrest works well. Using in a home doing backups to r2 and local s3.

Re: Go ahead, self-host Postgres

#146

Post by awestroke »

satvikpendem wrote: post #138
awestroke wrote: post #134

Earlier quoted context omitted.

But then you have to check that these mechanisms work regularly and manually

One thing I learned working in the industry, you have to check them when you're using AWS too.

Really? You're saying RDS backups can't be trusted?

Re: Go ahead, self-host Postgres

#147

Post by scott_w »

pinkgolem wrote: post #128
scott_w wrote: post #81

Earlier quoted context omitted.

From experience (because I used to do this), it’s a lot less time than a self-hosted solution, once you’re factoring in the multiple services that need to be maintained.

As someone who has done both.. i disagree, i find self hosting to a degree much easier and much less complex Local reproducibility is easier, and performance is often much better

It depends entirely on your use case. If all you need is a DB and Python/PHP/Node server behind Nginx then you can get away with that for a long time. Once you throw in a task runner, emails, queue systems, blob storage, user-uploaded content, etc. you can start running beyond your own ability or time to fix the inevitable problems.

As I pointed out above, you may be better served mixing and matching so you spend your time on the critical aspects but offload those other tasks to someone else.

Of course, I’m not sitting at your computer so I can’t tell you what’s right for you.

Re: Go ahead, self-host Postgres

#148

Post by chuckadams »

danaris wrote: post #88
paulryanrogers wrote: post #19

Earlier quoted context omitted.

IMO Maria has fallen behind MySQL. I wouldn't chose it for anything my income depends on. (I do use Maria at home for legacy reasons, and have used MySQL and Pg professionally for years.)

> IMO Maria has fallen behind MySQL. I wouldn't chose it for anything my income depends on. Can you give any details on that? I switched to MariaDB back in the day for my personal projects because (so far as I could tell) it was being updated more regularly, and it was more fully open source. (I don't recall offhand at this point whether MySQL switched to a fully paid model, or just less-open.)

One area where Maria lags significantly is JSON support. In MariaDB, JSON is just an alias for LONGTEXT plus validation: https://mariadb.com/docs/server/reference/data-types/string-...

Re: Go ahead, self-host Postgres

#149

Post by squeaky-clean »

pinkgolem wrote: post #137
scott_w wrote: post #80

Earlier quoted context omitted.

> Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. This doesn’t make sense as an argument. The reason the cloud is more complex is because that complexity is available. Under a certain size, a large number of cloud products simply can’t be managed in-house (and certainly not altogether). Als…

>At a smaller business I worked at, I was able to use these services to achieve uptime and performance that I couldn’t achieve self-hosted, because I had to spend time on the product itself. So yeah, we’d saved on infrastructure engineers. How sure are you about that one? All of my hetzner vm`s reach an uptime if 99.9% something. I could see more then one small business stack fitting onto a single of those vm`s.

Just because your VM is running doesn't mean the service is accessible. Whenever there's a large AWS outage it's usually not because the servers turned off. It also doesn't guarantee that your backups are working properly.

Re: Go ahead, self-host Postgres

#150

Post by satvikpendem »

awestroke wrote: post #146
satvikpendem wrote: post #138

Earlier quoted context omitted.

One thing I learned working in the industry, you have to check them when you're using AWS too.

Really? You're saying RDS backups can't be trusted?

Trusted in what sense, that they'll always work perfectly 100% of the time? No, therefore one must still check them from time to time, and it's really no different when self hosting, again, if you do it correctly.
Post reply on HN