Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

181–190 of 236 posts

Re: Ask HN: Do you self-host your database?

#181

Post by tut-urut-utut »

stronglikedan wrote: post #173
tut-urut-utut wrote: post #132

Earlier quoted context omitted.

But the thing is in most of the companies you don't have full control over the whole stack. Even if you have "full control" over the database, you don't have control over networking, firewall, OS, "security" patching, VMs, Docker, Kubernetes, Load balancers, vendors managing parts of the infra, internet provider, hosting provider ... Not even datacenter team may have control over all of it, but at least that's their…

> And receiving a call and telling: "here's datacenter emergency phone, please ping them" or forwarding mail to datacenter team with "can you give ETA to the business" is an order of magnitude easier than trying to concentrate and resolve the actual problem while being interrupted every 5 minutes and not having required level of access to actually do it, even if the knowledge exists. You seem to have exceptional cust…

That's a valid point, looks like we are not in the same role. I work mostly on internal software in a big "enterprise". My customers are mostly internal customers (marketing, sales, ...) but I have a luxury of having a business analyst, project manager or product owner that receive calls from them, so I don't need to explain anything directly to customers.

In rare cases that some C-level break through and complain directly to me, it's usually easy to explain that if he wants a solution or an estimate fast, he should be dealing with datacenter team, since I don't have any leverage upon them, and he can complain directly to the IT-Infrastructure Director, to whom I don't even have direct contact.

Re: Ask HN: Do you self-host your database?

#183

Post by caeril »

tut-urut-utut wrote: post #77

For personal use and side projects, I always self-host. It's so much cheaper considering the tiny size. I usually start with sqlite instead of the "real" database, and in 99% of the cases it stays so. At work, I never self-host. Life is so much easier if blame for whatever unrelated reason can be outsourced to some cloud provider or internal datacenter team. Because if we self host the database, we will be responsibl…

It doesn't work this way. You're also responsible for your poor hiring decisions, including third party providers.

There is no absolution in technical decisions, sorry.

> "have no idea, datacenter team is working on it. You can ask them?"

You can ask them? This response would get you fired from a Level 1 help desk in any competent organization, let alone a position of authority or execution.

Re: Ask HN: Do you self-host your database?

#184

Post by giovannibonetti »

codewithcheese wrote: post #86
ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

Slack used to partition their databases per workspace (customer) and then moved away from it. The "Disadvantages" section lists reasons not to do that. https://slack.engineering/scaling-datastores-at-slack-with-v... Two disadvantages that stand out for me are: 1. You must be able to scale the database up to your biggest customer and down to the your smallest this can be increasingly difficult to provision. 2. You are…

IIRC this is the kind of problem that CockroachDB seems to solve well, given their implementation of range partitioning with automatic rebalancing. This ensures the partitions are evenly distributed even with an uneven customer distribuiton.

I have no affiliation with the product, but I have been comparing alternatives and it seems like that's where it shines.

Re: Ask HN: Do you self-host your database?

#185

Post by closeparen »

teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

Chauffeur is the right model for most on-prem business IT, which is typically owned and physically held by the business, but driven by a contractor who comes to visit as needed.

Re: Ask HN: Do you self-host your database?

#186

Post by renewiltord »

andix wrote: post #105
codewithcheese wrote: post #86

Earlier quoted context omitted.

Slack used to partition their databases per workspace (customer) and then moved away from it. The "Disadvantages" section lists reasons not to do that. https://slack.engineering/scaling-datastores-at-slack-with-v... Two disadvantages that stand out for me are: 1. You must be able to scale the database up to your biggest customer and down to the your smallest this can be increasingly difficult to provision. 2. You are…

2. can be a huge security feature. If your web application only has access to the database of this one customer, you practically can't leak data from other customers by accident.

Slack has cross-workspace connectivity and communication as a product feature. It’s a really cool feature.

Re: Ask HN: Do you self-host your database?

#187

Post by renewiltord »

codewithcheese wrote: post #86
ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

Slack used to partition their databases per workspace (customer) and then moved away from it. The "Disadvantages" section lists reasons not to do that. https://slack.engineering/scaling-datastores-at-slack-with-v... Two disadvantages that stand out for me are: 1. You must be able to scale the database up to your biggest customer and down to the your smallest this can be increasingly difficult to provision. 2. You are…

This is a good post. Thanks for sharing!

Re: Ask HN: Do you self-host your database?

#188

Post by renewiltord »

Used to do this for myself and it worked great for over a decade. But because it was a decade old I had never made it easy to replicate. It was a pet.

I know better now, but also all of those other things are easier when they’re a terraform config than when they’re manually managed. I use RDS now.

Re: Ask HN: Do you self-host your database?

#189

Post by [deleted] »

teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

[deleted]

Re: Ask HN: Do you self-host your database?

#190

Post by moehm »

littlestymaar wrote: post #136
moehm wrote: post #110

Earlier quoted context omitted.

I think he means the usual stuff, so monitoring (disk space, memory, disk io, slow queries), patching (following mailing lists, emergency patch on sunday afternoon), failover, scaling (read replicas, multi master), backup, backup restore test, tuning, auth, security etc.

How does a cloud offering's going to help you with slow queries and tuning? Same for the security part (the biggest threats coming through your app anyway).

Honestly, I don't know and would like to know as well. Can you share some insights into how much a cloud database allows to be tuned?

I haven't used a cloud offering before, but OP was asking about managing and operating a database. And that list is what we do at work or at home.

Post reply on HN