Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 236 posts

Re: Ask HN: Do you self-host your database?

#211

Post by hsn915 »

benhurmarcel wrote: post #144
teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

And you don't usually grow your own food, yet it's not seen as strange.

It would be seen as strange if you say you are a farmer but you don't have a farm that you grow food in, instead you just temporarily rent a farm from random places for short amounts of time.

Last year I was renting my farm from AppleFarms but they started charging high prices when I wanted to expand so I switched to AmazonForestServices. For migration I used the DockerShippingService from GiantGarageInc to transplant all the seeds and plants from my previous farm to the new farm. They have amazing support and their staff was very diligent in plucking all the plants carefully.

Re: Ask HN: Do you self-host your database?

#213

Post by PikachuEXE »

vira28 wrote: post #12
PikachuEXE wrote: post #6

Self hosting PSQL via Docker container + Ansible. Also hosting MongoDB, Elastic Search, Redis instances (for session & caching, separate instances). Might consider researching using pg_bouncer later (but really not needed right now). Pricing and support for vendor solutions are too bad for small scale app.

> Pricing and support for vendor solutions are too bad for small scale app. In-line with my experience too. Where are some of the hard things that you learnt self hosting these many services?

PSQL no downtime upgrade (e.g. 12 > 13) via logical replication.

So difficult that I think it's worth writing an article for it: https://dev.to/pikachuexe/postgresql-logical-replication-for...

But staying update with latest version is possible and relatively easy with Docker: update the Dockerfile, build and push, run deploy via Ansible playbook

Re: Ask HN: Do you self-host your database?

#214

Post by zvr »

howaboutnope wrote: post #163

The idea of "outsourcing blame" mentioned in several comments seems really weird to me: If I make the choice to outsource something, and whoever I outsourced it to fucks up, I'm still the one who made the decision to outsource it. The same goes for outsourcing to someone who does a better job than I could have: then that was a great idea, and yay for me.

"Nobody ever got fired for buying IBM".

Re: Ask HN: Do you self-host your database?

#215

Post by gregw2 »

littlestymaar wrote: post #134
reacharavindh wrote: post #67

Earlier quoted context omitted.

As with most things in infrastructure, it is a tradeoff to be considered properly. A startup, where you'd rather have your engineers working on adding features to your product instead of "managing and operating" a database? Have a lot of money to throw at this problem, being aware that you are paying for convenience? then using RDS and such would probably be a better choice. An established company/product with a team…

> your engineers working on adding features to your product instead of "managing and operating" a database? That's the typical sales pitch for cloud offering, but I really fails to see how it applies to DB. Which part of “managing and operating” your database do you save when using such a service? And how big of a load did it put on your engineers in the first place?

Your crown jewels backup/restore will always work. You can easily/always spin up a new db copy.

Re: Ask HN: Do you self-host your database?

#216

Post by exikyut »

mceachen wrote: post #208
tyingq wrote: post #166

Earlier quoted context omitted.

SQLite mentions NFS as problematic because SQLite makes a lot of use of advisory locking. Which is supposed to work on NFS, but apparently is often buggy.

Writing to a SQLite file on a remote filesystem only works with a heap of caveats: don't use WAL, and disable SHM. SQLite is crazy fast in WAL mode, but it's because they use a memory-mapped file to coordinate consistency across multiple processes/threads. You can't mmap a remote file. (Source: hacking in a DB replica workaround to PhotoStructure so people can store their library on arbitrary filesystems... BTW, I'd…

I'm idly curious why you need to use replicas in a workaround context, and why arbitrary filesystems are a problem.

Re: Ask HN: Do you self-host your database?

#217

Post by exikyut »

cutthegrass2 wrote: post #71
ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

This sounds super interesting, are you able to share any further detail? I'm currently wrestling with a 35TB Aurora DB and considering options for future growth. The system is multi-tenant and one of the options on the table is breaking out the tenants and hosting them on either their own system or alongside smaller subset of tenants.

I must admit I'm very curious as to what real-world thing(s) translate to 35TB of stored data (that presumably isn't large blobs, like video or ML models).

(And oh wow I just realized the billing must be amazing...)

Re: Ask HN: Do you self-host your database?

#218

Post by reacharavindh »

littlestymaar wrote: post #134
reacharavindh wrote: post #67

Earlier quoted context omitted.

As with most things in infrastructure, it is a tradeoff to be considered properly. A startup, where you'd rather have your engineers working on adding features to your product instead of "managing and operating" a database? Have a lot of money to throw at this problem, being aware that you are paying for convenience? then using RDS and such would probably be a better choice. An established company/product with a team…

> your engineers working on adding features to your product instead of "managing and operating" a database? That's the typical sales pitch for cloud offering, but I really fails to see how it applies to DB. Which part of “managing and operating” your database do you save when using such a service? And how big of a load did it put on your engineers in the first place?

As a Sysadmin who has spent a lot of time engineering systems to do specific tasks, I will tell you that there is a lot of effort that goes into your "production" database. Some of the tasks are not going to magically go away just because you use a managed cloud service for database - securing access to your database, monitoring basic metrics such as CPU, disk usage etc, making sure your data is backed up and tested for recovery. However, if you decide to self-host your production database, you've got more work such as

1. Keeping your database software up-to-date. Applying security patches without a huge downtime.

2. Managing the installation so that it is operating correctly - it's Systemd init files are okay and that it will come back to life should the server restart, logs are being stored correctly at an appropriate disk partition and that they don't fill up your data directory,

3. OS configuration aligns with your and your database software's needs. Which sysenvs did you set? were they changed as part of your OS updates?

All of the above are worth it in many cases. Spending a few days optimising the installation for your workload can make a huge difference and also empower you to use custom extensions like Timescale which is not so easy to do in some managed database services. Also, cost savings because you take that effort on yourself.

Re: Ask HN: Do you self-host your database?

#219

Post by vanviegen »

jdhawk wrote: post #174
vanviegen wrote: post #143

For self-hosting, MariaDB (or Percona or MySQL) with InnoDB is really unbeatable. - Version upgrades don't require migrations. - Replication is pretty easy, well understood and allows version differences between the two ends. Nowadays they even got crash resilience right. - It doesn't require VACUUM or any other regular maintenance. - XtraBackup is awesome. I've been running a pretty large and rather critical MariaDB…

Are you using GTID's for replication? Most of my replication issues stem from not being able to use them

Yes, as I understand it GTIDs are part of what enables replication safe crashes.

Re: Ask HN: Do you self-host your database?

#220

Post by jenkstom »

mgkimsal wrote: post #123
tut-urut-utut wrote: post #77

For personal use and side projects, I always self-host. It's so much cheaper considering the tiny size. I usually start with sqlite instead of the "real" database, and in 99% of the cases it stays so. At work, I never self-host. Life is so much easier if blame for whatever unrelated reason can be outsourced to some cloud provider or internal datacenter team. Because if we self host the database, we will be responsibl…

> Just "have no idea, datacenter team is working on it. You can ask them?" has done wonders for my mental health and job satisfaction Never seems to have much benefit for me, because I'm typically the one who answers the phone/email/etc, and "the datacenter team" never does. No one is happy with that, and I still get the calls/email/messages, demanding the 'fix' or ETA. I would rather things be more under my control,…

Set and maintain strong, healthy adult boundaries. If somebody blames you for something that you don't control then call them out on it. You don't even have to be mean, just point it out. There's a lot of legal precedence for "But (s)he allowed me to abuse him(her)!" as an excuse for abuse.

Honestly that's the best thing you can do for on-the-job sanity. I understand that people feel they have a right to abuse others, but at some point you really don't have to take it.

Post reply on HN