Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

171–180 of 407 posts

Re: Go ahead, self-host Postgres

#171

Post by [deleted] »

molf wrote: post #49
Nextgrid wrote: post #42

Earlier quoted context omitted.

> employing engineers to manage self-hosted databases is more cost effective than outsourcing Every company out there is using the cloud and yet still employs infrastructure engineers to deal with its complexity. The "cloud" reducing staff costs is and was always a lie. PaaS platforms (Heroku, Render, Railway) can legitimately be operated by your average dev and not have to hire a dedicated person; those cost even mo…

> Every company out there is using the cloud and yet still employs infrastructure engineers Every company beyond a particular size surely? For many small and medium sized companies hiring an infrastructure team makes just as little sense as hiring kitchen staff to make lunch.

[deleted]

Re: Go ahead, self-host Postgres

#172

Post by pinkgolem »

scott_w wrote: post #147
pinkgolem wrote: post #128

Earlier quoted context omitted.

As someone who has done both.. i disagree, i find self hosting to a degree much easier and much less complex Local reproducibility is easier, and performance is often much better

It depends entirely on your use case. If all you need is a DB and Python/PHP/Node server behind Nginx then you can get away with that for a long time. Once you throw in a task runner, emails, queue systems, blob storage, user-uploaded content, etc. you can start running beyond your own ability or time to fix the inevitable problems. As I pointed out above, you may be better served mixing and matching so you spend you…

I mean, fair, we are ofc offloading some of that.. email being one of those, LLM`s being another thing.

Task runner/que at least for us postgres works for both cases.

We also self host an s3 storage and allow useruploaded content in within strict borders.

Re: Go ahead, self-host Postgres

#173

Post by kachapopopow »

chandureddyvari wrote: post #110
kachapopopow wrote: post #93

since this is on the front page (again?) I guess I'll chime in: learn kubernetes - it's worth it. It did take me 3 attempts at it to finally wrap my head around it I really suggest trying out many different things and see what works for you. And I really recommend starting with *default* k3s, do not look at any alternatives to cni, csi, networked storage - treat your first cluster as something that can spontaniously…

Any good recommendations you got for learning kubernetes for busy people?

k3sup a cluster, ask an AI on how to serve an nginx static site using trafeik on it and explain every step of it and what it does (it should provide: a config map, a deployment, a service and an ingress)

k3s provides: csi, cni (cluster storage interface, cluster network interface) which is flannel and and local-pv which just maps volumes to disk (pvcs)

trafeik is what routes your traffic from the outside to inside your cluster (to an ingress resource)

Re: Go ahead, self-host Postgres

#174

Post by alemanek »

paulryanrogers wrote: post #160
mattmanser wrote: post #136

Earlier quoted context omitted.

Over 20 year I've had lots of clients on self-hosted, even self-hosting SQL on the same VM as the webserver as you used to in the long distant past for low-usage web apps. I have never, ever, ever had a SQL box go down. I've had a web server go down once. I had someone who probably shouldn't have had access to a server accidentally turn one off once. The only major outage I've had (2/3 hours) was when the box was als…

Disks go bad. RAID is nontrivial to set up. Hetzner had a big DC outage that lead to data loss. Off site backups or replication would help, though not always trivial to fail over.

As someone who has set this up while not being a DBA or sysadmin.

Replication and backups really aren’t that difficult to setup properly with something like Postgres. You can also expose metrics around this to setup alerting if replication lag goes beyond a threshold you set or a backup didn’t complete. You do need to periodically test your backups but that is also good practice.

I am not saying something like RDS doesn’t have value but you are paying a huge premium for it. Once you get to more steady state owning your database totally makes sense. A cluster of $10-20 VPSes with NVMe drives can get really good performance and will take you a lot farther than you might expect.

Re: Go ahead, self-host Postgres

#175

Post by mystifyingpoi »

wreath wrote: post #124

> Take AWS RDS. Under the hood, it's: Standard Postgres compiled with some AWS-specific monitoring hooks A custom backup system using EBS snapshots Automated configuration management via Chef/Puppet/Ansible Load balancers and connection pooling (PgBouncer) Monitoring integration with CloudWatch Automated failover scripting I didn't know RDS had PgBouncer under the hood, is this really accurate? The problem i find wit…

> I didn't know RDS had PgBouncer under the hood

I don't think it does. AWS has this feature under RDS Proxy, but it's an extra service and comes with extra cost (and a bit cumbersome to use in my opinion, it should have been designed as a checkbox, not an entire separate thing to maintain).

Although, it technically has "load balancer", in form of a DNS entry that resolves to a random reader replica, if I recall correctly.

Re: Go ahead, self-host Postgres

#176

Post by Nextgrid »

flomo wrote: post #157
add-sub-mul-div wrote: post #72

Earlier quoted context omitted.

You're paying people to do the role either way, if it's not dedicated staff then it's taking time away from your application developers so they can play the role of underqualified architects, sysadmins, security engineers.

Yeah, and nobody is looking at the other side of this. There just are not a lot of good DBA/sysop type who even want to work for some non-tech SMB. So this either gets outsourced to the cloud, or some junior dev or desktop support guy hacks it together. And then who knows if the backups are even working. Fact is a lot of these companies are on the cloud because their internal IT was a total fail.

If they just paid half of the markup they currently pay for the cloud I'm sure they'll be swimming in qualified candidates.

Re: Go ahead, self-host Postgres

#177

Post by pinkgolem »

squeaky-clean wrote: post #149
pinkgolem wrote: post #137

Earlier quoted context omitted.

>At a smaller business I worked at, I was able to use these services to achieve uptime and performance that I couldn’t achieve self-hosted, because I had to spend time on the product itself. So yeah, we’d saved on infrastructure engineers. How sure are you about that one? All of my hetzner vm`s reach an uptime if 99.9% something. I could see more then one small business stack fitting onto a single of those vm`s.

Just because your VM is running doesn't mean the service is accessible. Whenever there's a large AWS outage it's usually not because the servers turned off. It also doesn't guarantee that your backups are working properly.

If you have a server where everything is on the server, the server being on means everything is online... There is not a lot of complexity going on inside a single server infrastructure.

I mean just because you have backups does not mean you can restore them ;-)

We do test backup restoration automatically and also on a quarterly basis manually, but so you should do with AWS.

Otherwise how do you know you can restore system a without impact other dependency, d and c

Re: Go ahead, self-host Postgres

#178

Post by esseph »

emseetech wrote: post #103
molf wrote: post #36

> I'd argue self-hosting is the right choice for basically everyone, with the few exceptions at both ends of the extreme: > If you're just starting out in software & want to get something working quickly with vibe coding, it's easier to treat Postgres as just another remote API that you can call from your single deployed app > If you're a really big company and are reaching the scale where you need trained database e…

| self hosting costs you between 30 and 120 minutes per month Can we honestly say that cloud services taking a half hour to two hours a month of someone's time on average is completely unheard of?

Very much depends on what you're doing in the cloud, how many services you are using, and how frequently those services and your app needs updates.

Re: Go ahead, self-host Postgres

#179

Post by pinkgolem »

scott_w wrote: post #140
pinkgolem wrote: post #137

Earlier quoted context omitted.

>At a smaller business I worked at, I was able to use these services to achieve uptime and performance that I couldn’t achieve self-hosted, because I had to spend time on the product itself. So yeah, we’d saved on infrastructure engineers. How sure are you about that one? All of my hetzner vm`s reach an uptime if 99.9% something. I could see more then one small business stack fitting onto a single of those vm`s.

100% certain because I started by self hosting before moving to AWS services for specific components and improved the uptime and reduced the time I spent keeping those services alive.

What was work you spend configuring those services and keeping them alive? I am genuinely curious...

We have a very limited set of services, but most have been very painless to maintain.

Re: Go ahead, self-host Postgres

#180

Post by graemep »

ZeroConcerns wrote: post #11

So, yeah, I guess there's much confusion about what a 'managed database' actually is ? Because for me, the table stakes are: -Backups: the provider will push a full generic disaster-recovery backup of my database to an off-provider location at least daily, without the need for a maintenance window -Optimization: index maintenance and storage optimization are performed automatically and transparently -Multi-datacenter…

Which providers do all of that?
Post reply on HN