Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 236 posts

Re: Ask HN: Do you self-host your database?

#141

Post by roland35 »

teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

I think the main point is self managed vs buying the service? I think a more apt metaphor might be renting a car from a company who takes care of the car vs buying a car and managing all the maintenance myself.

Re: Ask HN: Do you self-host your database?

#142

Post by ngrilly »

simonw wrote: post #54
ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

I'd be fascinated to hear more about this. Where do the SQLite files live? Are you using some kind of NFS or EFS or similar for them? Sharding them across many machines? How are you handling backups and high availability? Are you using Litestream? How many SQLite files do your application servers have open at any one time?

Also super interested in hearing more.

Re: Ask HN: Do you self-host your database?

#143

Post by vanviegen »

For self-hosting, MariaDB (or Percona or MySQL) with InnoDB is really unbeatable.

- Version upgrades don't require migrations.

- Replication is pretty easy, well understood and allows version differences between the two ends. Nowadays they even got crash resilience right.

- It doesn't require VACUUM or any other regular maintenance.

- XtraBackup is awesome.

I've been running a pretty large and rather critical MariaDB database for about 15 years (of course it has migrated to a different machine a couple of times), without anything 'interesting' happening. (Except for power failures messing up replication consistency - but that seems to be a thing of the past.)

My experiences managing PostgreSQL were... not as great. (Though with regard to most other aspects, PostgreSQL is a lot nicer than MariaDB.)

Re: Ask HN: Do you self-host your database?

#144

Post by benhurmarcel »

teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

And you don't usually grow your own food, yet it's not seen as strange.

Re: Ask HN: Do you self-host your database?

#145

Post by WFHRenaissance »

teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

Internet speeds may suck in some parts of the US, but don't get it twisted: companies host their DBs on the cloud to mitigate the risk of their own incompetence/for lean gains from a managed service (RDS has done wonders in this realm).

Re: Ask HN: Do you self-host your database?

#146

Post by torgard »

tester34 wrote: post #81
reacharavindh wrote: post #67

Earlier quoted context omitted.

As with most things in infrastructure, it is a tradeoff to be considered properly. A startup, where you'd rather have your engineers working on adding features to your product instead of "managing and operating" a database? Have a lot of money to throw at this problem, being aware that you are paying for convenience? then using RDS and such would probably be a better choice. An established company/product with a team…

What do you mean by "managing and operating" a database? Applying migrations? initial setup and configuration?

> initial setup and configuration?

If you think there is nothing more to maintaining a database than initial setup and configuration, you are in for a rude awakening. Shit happens.

Re: Ask HN: Do you self-host your database?

#147

Post by axegon_ »

For work projects - no, haven't in many years, since 2016 iirc. For personal projects - yes - I have plenty of hardware resources to spare so it costs as much as the electricity needed to run them(which they do regardless). That said, backup dumps go to a gcp bucket since it's far less likely to fail then one or two drives shoved in the basement and storage itself is incredibly cheap. Besides personal projects in my case are not a source of income so cutting costs as much as possible is the sensible thing to do.

Re: Ask HN: Do you self-host your database?

#148

Post by tehbeard »

jrochkind1 wrote: post #139
ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

> Keeping the PG servers up to date is a pain, we simply don't have time for this I've never used AWS RDS (or similar), but I had been thinking that part of the advantage of RDS was you didn't need to "keep the PG server up to date". But apparently I was wrong! What does this look like on RDS?

(note: we use MySQL on RDS, but the docs for PG seem to be similiar enough)

Minor versions you can tell it a day and rough timeslot (few hours) for it to apply them. Major version upgrades always require you to start them, and the docs lay out prep work to do before hand (check for incompatibilities, setup any custom config before hand, take a backup/snapshot, do dry run)

One of the things I love with RDS is that last bit, doing a dry run with a clone of the database is fairly easy (can't say what it's like at the TB level of data, probably not as easy/quick) to test.

Re: Ask HN: Do you self-host your database?

#149

Post by lbriner »

I'm not sure whether you are asking whether people use Database-as-a-service instead of installing the DB themselves or whether you are asking about on-premise vs cloud Infrastructure-as-a-service.

As many have noted, DaaS is very convenient but not always completely flexible with configuration and the pricing is not comparable since there is usually a minimum price even if you only want 1 table with 10 rows so it won't scale in the same way.

For all of my home and work projects, we have servers hosted on the cloud to get better internet bandwidth and we install and run our own SQL Server and MySQL instances. A bare Vm is pretty cheap on the cloud and installing databases is fairly easy, although I have never personally setup clustering or failover in MySQL or SQL Server, I think that is quite involved.

Re: Ask HN: Do you self-host your database?

#150

Post by cutthegrass2 »

gregwebs wrote: post #106
cutthegrass2 wrote: post #71

Earlier quoted context omitted.

This sounds super interesting, are you able to share any further detail? I'm currently wrestling with a 35TB Aurora DB and considering options for future growth. The system is multi-tenant and one of the options on the table is breaking out the tenants and hosting them on either their own system or alongside smaller subset of tenants.

The issue with this approach (other than isolation) is that the largest tenant may be orders of magnitude bigger than others and still need more sharding to perform well while their smallest still needs to share a db. Have you looked at TiDB (MySQL) or CRDB (PG)?

Absolutely.

Thanks for the suggestion of TiDB, haven't previously considered this but it looks interesting, will take a closer look.

Post reply on HN