Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go ahead, self-host Postgres

pierce.dev

Post reply on HN
Print view View original

161–170 of 407 posts

Re: Go ahead, self-host Postgres

#161

Post by vovavili »

ninkendo wrote: post #118
kachapopopow wrote: post #93

since this is on the front page (again?) I guess I'll chime in: learn kubernetes - it's worth it. It did take me 3 attempts at it to finally wrap my head around it I really suggest trying out many different things and see what works for you. And I really recommend starting with *default* k3s, do not look at any alternatives to cni, csi, networked storage - treat your first cluster as something that can spontaniously…

At $WORK we’ve been using the Zalando Postgres kubernetes operator to great success: https://github.com/zalando/postgres-operator As someone who has operated Postgres clusters for over a decade before k8s was even a thing, I fully recommend just using a Postgres operator like this one and moving on. The out of box config is sane, it’s easy to override things, and failover/etc has been working flawlessly for years. It…

For something like a database, what is the added advantage to using Kubernetes as opposed to something simple like Docker Compose?

Re: Go ahead, self-host Postgres

#162

Post by olavgg »

alexfromapex wrote: post #123

Just don't try to build it from source haha. Compiling Postgres 18 with the PostGIS extension has been such a PITA because the topology component won't configure to not use the system /usr/bin/postgres and has given me a lot of grief. Finally got it fixed I think though.

I actually always build PostgreSQL from source as I want 32kb block size as default. It makes ZFS compression more awesome.

Re: Go ahead, self-host Postgres

#163

Post by darksaints »

honestly at this point I'm actually surprised that there aren't specialized linux distributions for hosting postgres. There's so many kernel-level and file-system level optimizations that can be done that significantly impact performance, and the ability to pare down all of the unneeded stuff in most distributions would make for a pretty compact and highly optimized image.

Re: Go ahead, self-host Postgres

#164

Post by NewJazz »

Nextgrid wrote: post #83
convolvatron wrote: post #40

Earlier quoted context omitted.

its not. I've been in a few shops that use RDS because they think their time is better spend doing other things. except now they are stuck trying to maintain and debug Postgres without having the same visibility and agency that they would if they hosted it themselves. situation isn't at all clear.

One thing unaccounted for if you've only ever used cloud-hosted DBs is just how slow they are compared to a modern server with NVME storage. This leads the developers to do all kinds of workarounds and reach for more cloud services (and then integrating them and - often poorly - ensuring consistency across them) because the cloud hosted DB is not able to handle the load. On bare-metal, you can go a very long way with…

Yeah our cloud DBs all have abysmal performance and high recurring cost even compared to metal we didn't even buy for hosting DBs.

Re: Go ahead, self-host Postgres

#165

Post by olavgg »

adenta wrote: post #51

I wish this article would have went more in-depth on how they're setting up backups. The great thing about sequel light is lightstream makes backup and restore something you don't really have to think about

ZFS snapshot, send, receive, clone, spin up another postgresql server on the backup server, take full backup on that clone once per week

Re: Go ahead, self-host Postgres

#166

Post by mind-blight »

I think a big piece missing from these conversations is compliance frameworks and customer trust. Of your selling to enterprise customers or governments, they want to go through your stack, networking, security, audit logs, and access controls with a fine toothed comb.

Everything you do that isn't "normal" is another conversation you need to have with an auditor plus each customer. Those eat up a bunch of time and deals take longer to close.

Right or wrong, these decisions make you less "serious" and therefore less credible in the eyes of many enterprise customers. You can get around that perception, but it takes work. Not hosting on one of the big 3 needs to be decided with that cost in mind

Re: Go ahead, self-host Postgres

#167

Post by yomismoaqui »

Now for the next step... just use SQLite (it's possible it will be enough for your case).

Disclaimer: there's no silver bullet, yadda yadda. But SQLite in WAL mode and backups using Litestream have worked perfectly for me.

Re: Go ahead, self-host Postgres

#168

Post by mystifyingpoi »

vovavili wrote: post #161
ninkendo wrote: post #118

Earlier quoted context omitted.

At $WORK we’ve been using the Zalando Postgres kubernetes operator to great success: https://github.com/zalando/postgres-operator As someone who has operated Postgres clusters for over a decade before k8s was even a thing, I fully recommend just using a Postgres operator like this one and moving on. The out of box config is sane, it’s easy to override things, and failover/etc has been working flawlessly for years. It…

For something like a database, what is the added advantage to using Kubernetes as opposed to something simple like Docker Compose?

Docker Compose (ignoring Swarm which seems to be obsolete) manages containers on a single machine. With Kubernetes, the pod that hosts the database is a pod like any other (I assume). It gets moved to a healthy machine when node goes bad, respects CPU/mem limits, works with generic monitoring tools, can be deployed from GitOps tools etc. All the k8s goodies apply.

Re: Go ahead, self-host Postgres

#169

Post by kachapopopow »

vovavili wrote: post #161
ninkendo wrote: post #118

Earlier quoted context omitted.

At $WORK we’ve been using the Zalando Postgres kubernetes operator to great success: https://github.com/zalando/postgres-operator As someone who has operated Postgres clusters for over a decade before k8s was even a thing, I fully recommend just using a Postgres operator like this one and moving on. The out of box config is sane, it’s easy to override things, and failover/etc has been working flawlessly for years. It…

For something like a database, what is the added advantage to using Kubernetes as opposed to something simple like Docker Compose?

I hate that this is starting to sound like a bot Q&A, but the primary advantages is that it provides secure remote configuration and it's that it's platform agnostic, multi-node orchestration, built in load balancing and services framework, way more networking control than docker, better security, self healing and the list goes on, you have to read more about it to really understand the advantages over docker.

Re: Go ahead, self-host Postgres

#170

Post by mystifyingpoi »

chandureddyvari wrote: post #110
kachapopopow wrote: post #93

since this is on the front page (again?) I guess I'll chime in: learn kubernetes - it's worth it. It did take me 3 attempts at it to finally wrap my head around it I really suggest trying out many different things and see what works for you. And I really recommend starting with *default* k3s, do not look at any alternatives to cni, csi, networked storage - treat your first cluster as something that can spontaniously…

Any good recommendations you got for learning kubernetes for busy people?

No path for busy people, unfortunately. Learn everything from ground up, from containers to Compose to k3s, maybe to kubeadm or hosted. Huge abstraction layers coming from Kubernetes serve their purpose well, but can screw you up when anything goes slightly wrong on the upper layer.

For start, ignore operators, ignore custom CSI/CNI, ignore IAM/RBAC. Once you feel good in the basics, you can expand.

Post reply on HN