Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 236 posts

Re: Ask HN: Do you self-host your database?

#71

Post by cutthegrass2 »

ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

This sounds super interesting, are you able to share any further detail?

I'm currently wrestling with a 35TB Aurora DB and considering options for future growth. The system is multi-tenant and one of the options on the table is breaking out the tenants and hosting them on either their own system or alongside smaller subset of tenants.

Re: Ask HN: Do you self-host your database?

#72

Post by lkuty »

weavie wrote: post #60
vira28 wrote: post #32

Earlier quoted context omitted.

Amazing. What are some advices that you would give for someone who is looking to self host?

Never run postgres with the default configuration. It comes with some extremely conservative settings so you are unlikely to get decent performance out of the box.

Could you elaborate on this or point us to documentation (url, book, ...) about it ?

Re: Ask HN: Do you self-host your database?

#73

Post by latch »

We host our own CockroachDB cluster on 3 baremetal machines.

We also host 2 separate PostgreSQL instances, each with an asynchronous replica (which serve some of the read queries) and with bareman for PITR.

The PG instances used to be on RDS. The performance difference (and cost savings) is....at least least 10x.

We have no devops/sysadmins.

Re: Ask HN: Do you self-host your database?

#74

Post by strzibny »

geostar wrote: post #66
vbsteven wrote: post #30

Yes, I selfhost Postgres/Redis/Mongo for small projects (db + app server on the same machine or a small number of VMs). Usually in docker, sometimes systemd units. It's amazing what you can do on a single $5-50/month machine if HA isn't super important. On larger projects (typically once k8s gets involved) I'm running on a cloud provider anyway and I might as well use a hosted version like RDS for the main database.…

Do you prefer docker over systemd units?

I am not OP, but I prefer systemd unit myself. I can use stock CentOS/Rocky/RHEL/Fedora, install the database package and be running in no time.

Re: Ask HN: Do you self-host your database?

#76

Post by weavie »

lkuty wrote: post #72
weavie wrote: post #60

Earlier quoted context omitted.

Never run postgres with the default configuration. It comes with some extremely conservative settings so you are unlikely to get decent performance out of the box.

Could you elaborate on this or point us to documentation (url, book, ...) about it ?

The postgres docs are excellent, https://www.postgresql.org/docs/13/runtime-config.html

I have seen tutorials where folks go over the most essential settings, but I can't seem to find the ones I have used now, so me just doing a search for 'postgres tuning' is no better than you doing the search yourself.

Maybe someone else can chime in with a good guide?

Re: Ask HN: Do you self-host your database?

#77

Post by tut-urut-utut »

For personal use and side projects, I always self-host. It's so much cheaper considering the tiny size. I usually start with sqlite instead of the "real" database, and in 99% of the cases it stays so.

At work, I never self-host. Life is so much easier if blame for whatever unrelated reason can be outsourced to some cloud provider or internal datacenter team.

Because if we self host the database, we will be responsible that it is not reachable, even if it's because someone in datacenter changed some seemingly unrelated firewall rule. I want to avoid having to explain anything to our customers. Just "have no idea, datacenter team is working on it. You can ask them?" has done wonders for my mental health and job satisfaction.

Re: Ask HN: Do you self-host your database?

#78

Post by thorin »

A lot of big companies do. The large UK utility I work for has dozens of large production Oracle dbs. I'm sure this is also the case with SQLServer / Postgres / DB2(!). The tide is turning though and for new systems /projects without strict regulatory requirements I'm sure cloud will become the primary choice.

Re: Ask HN: Do you self-host your database?

#79

Post by scottydelta »

I am self-hosting postgres containerized on the server with auto encrypted backups to s3.

Besides this I also run a self hosted production grade elasticsearch cluster which I manage via ansible.

Both of these would be insanely expensive if I chose any of the managed solutions from aws or gcp.

Post reply on HN