Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 236 posts

Re: Ask HN: Do you self-host your database?

#131

Post by hypertele-Xii »

eric4smith wrote: post #17

IMHO self hosting your database (even in the cloud) is the best way to do it. You have control over the version. You have control over features. You have control over performance. It’s tons cheaper for greater performance - especially when you go over a few hundred gigs. Yes, the hosted ones have built in replication - but my data is far too valuable to put in the hands of a third party. If they lost it - they could…

And every thing you have control over you are also responsible for maintaining. If that's your thing, ok, but not everyone wants to be a DBA all day.

Re: Ask HN: Do you self-host your database?

#132

Post by tut-urut-utut »

mgkimsal wrote: post #123
tut-urut-utut wrote: post #77

For personal use and side projects, I always self-host. It's so much cheaper considering the tiny size. I usually start with sqlite instead of the "real" database, and in 99% of the cases it stays so. At work, I never self-host. Life is so much easier if blame for whatever unrelated reason can be outsourced to some cloud provider or internal datacenter team. Because if we self host the database, we will be responsibl…

> Just "have no idea, datacenter team is working on it. You can ask them?" has done wonders for my mental health and job satisfaction Never seems to have much benefit for me, because I'm typically the one who answers the phone/email/etc, and "the datacenter team" never does. No one is happy with that, and I still get the calls/email/messages, demanding the 'fix' or ETA. I would rather things be more under my control,…

But the thing is in most of the companies you don't have full control over the whole stack. Even if you have "full control" over the database, you don't have control over networking, firewall, OS, "security" patching, VMs, Docker, Kubernetes, Load balancers, vendors managing parts of the infra, internet provider, hosting provider ... Not even datacenter team may have control over all of it, but at least that's their job and their area of expertise.

And receiving a call and telling: "here's datacenter emergency phone, please ping them" or forwarding mail to datacenter team with "can you give ETA to the business" is an order of magnitude easier than trying to concentrate and resolve the actual problem while being interrupted every 5 minutes and not having required level of access to actually do it, even if the knowledge exists.

Though in most of the companies, business will not call you, but some poor business analyst or project/product manager, which will learn quickly to keep you out of the loop and don't waste anyone's time if you are consistent with answering "have no idea, ask datacenter guys".

Re: Ask HN: Do you self-host your database?

#133

Post by laurent92 »

codewithcheese wrote: post #86
ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

Slack used to partition their databases per workspace (customer) and then moved away from it. The "Disadvantages" section lists reasons not to do that. https://slack.engineering/scaling-datastores-at-slack-with-v... Two disadvantages that stand out for me are: 1. You must be able to scale the database up to your biggest customer and down to the your smallest this can be increasingly difficult to provision. 2. You are…

3. By splitting, you can give your customers control on data residency, as mandatory in GDPR,

4. By splitting, you limit the exposure in case of SQL injection.

5. By splitting, you can have good performance for the customer who has 1M records, and good performance for the 100K customers with 10 records.

Re: Ask HN: Do you self-host your database?

#134

Post by littlestymaar »

reacharavindh wrote: post #67
eric4smith wrote: post #17

IMHO self hosting your database (even in the cloud) is the best way to do it. You have control over the version. You have control over features. You have control over performance. It’s tons cheaper for greater performance - especially when you go over a few hundred gigs. Yes, the hosted ones have built in replication - but my data is far too valuable to put in the hands of a third party. If they lost it - they could…

As with most things in infrastructure, it is a tradeoff to be considered properly. A startup, where you'd rather have your engineers working on adding features to your product instead of "managing and operating" a database? Have a lot of money to throw at this problem, being aware that you are paying for convenience? then using RDS and such would probably be a better choice. An established company/product with a team…

> your engineers working on adding features to your product instead of "managing and operating" a database?

That's the typical sales pitch for cloud offering, but I really fails to see how it applies to DB. Which part of “managing and operating” your database do you save when using such a service? And how big of a load did it put on your engineers in the first place?

Re: Ask HN: Do you self-host your database?

#136

Post by littlestymaar »

moehm wrote: post #110
tester34 wrote: post #81

Earlier quoted context omitted.

What do you mean by "managing and operating" a database? Applying migrations? initial setup and configuration?

I think he means the usual stuff, so monitoring (disk space, memory, disk io, slow queries), patching (following mailing lists, emergency patch on sunday afternoon), failover, scaling (read replicas, multi master), backup, backup restore test, tuning, auth, security etc.

How does a cloud offering's going to help you with slow queries and tuning? Same for the security part (the biggest threats coming through your app anyway).

Re: Ask HN: Do you self-host your database?

#137

Post by teddyh »

DataBrokerLol wrote: post #129
teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

Can you provide some more information about the claim that the US has abysmal internet speeds and capacity? Edit: Grammar.

I said “abysmal private internet speeds and capacity”. Note the word “private” (which you omitted). By all accounts, the speed and capacity available to US residents in their homes is far inferior (both in price and speed/capacity) to what is available in many nations in the world. Sorry, I don’t have a reference, it’s just what I hear people say every time the topic comes up.

Re: Ask HN: Do you self-host your database?

#138

Post by juangacovas »

vira28 wrote: post #9
juangacovas wrote: post #5

Self-hosting MariaDB with multi-master replication. Works pretty well for my cases. Have a friend that was surprised by the AWS pricing changes on Aurora (they charge per query now instead of bandwidth/usage?)

multi-master with conflict resolution is non-trivial. Do you have automated failover set up? Also, didn't know about Aurora's per query pricing. That is crazy.

Sorry, I meant multi-source replication (different masters, different DBs to same slave(s)).

Re: Ask HN: Do you self-host your database?

#139

Post by jrochkind1 »

ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

> Keeping the PG servers up to date is a pain, we simply don't have time for this

I've never used AWS RDS (or similar), but I had been thinking that part of the advantage of RDS was you didn't need to "keep the PG server up to date".

But apparently I was wrong! What does this look like on RDS?

Re: Ask HN: Do you self-host your database?

#140

Post by Black101 »

DataBrokerLol wrote: post #129
teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

Can you provide some more information about the claim that the US has abysmal internet speeds and capacity? Edit: Grammar.

Not only it is on average slower in the US, but also usually more expensive: https://fairinternetreport.com/research/internet-speed-by-co... (first link I found doing a web search)
Post reply on HN