Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 236 posts

Re: Ask HN: Do you self-host your database?

#121

Post by tut-urut-utut »

PaulKeeble wrote: post #100
tut-urut-utut wrote: post #77

For personal use and side projects, I always self-host. It's so much cheaper considering the tiny size. I usually start with sqlite instead of the "real" database, and in 99% of the cases it stays so. At work, I never self-host. Life is so much easier if blame for whatever unrelated reason can be outsourced to some cloud provider or internal datacenter team. Because if we self host the database, we will be responsibl…

I definitely have a preference to have my own database for the purpose of development so I can change the structure, data and such as I need. It is kind of a necessity for developing a good suite of functional tests. All too often I have seen companies share a development database and that is a really bad sign for testability and generally stepping on each other. So if that is managed elsewhere but I can do all I nee…

That's definitely true. For development purpose I also prefer a local database that is only mine, but for production, staging, ... I don't want even to have access. Because in many companies, "you touch it, you own it" rule exists.

Re: Ask HN: Do you self-host your database?

#122

Post by bsenftner »

I embed an SqlLite3 DB into the application, which self hosts the DB as well as self hosts it's own Internet/web server. The application syncs with other copies of itself running in other locations. The application hosts its own DB, its own server, and has a RAM footprint of under 2MB before the DB is loaded into the memory mapped backing file. The application (video security) is screamingly fast, cross platform, and 100% self contained. Works fine on an air gapped network. Its biggest issue is the hardware footprint and expense to operate is so much lower than every other option, it makes people suspicious. That suspicion should be on the crappy other options, so bloated they are a sad, expensive joke.

Re: Ask HN: Do you self-host your database?

#123

Post by mgkimsal »

tut-urut-utut wrote: post #77

For personal use and side projects, I always self-host. It's so much cheaper considering the tiny size. I usually start with sqlite instead of the "real" database, and in 99% of the cases it stays so. At work, I never self-host. Life is so much easier if blame for whatever unrelated reason can be outsourced to some cloud provider or internal datacenter team. Because if we self host the database, we will be responsibl…

> Just "have no idea, datacenter team is working on it. You can ask them?" has done wonders for my mental health and job satisfaction

Never seems to have much benefit for me, because I'm typically the one who answers the phone/email/etc, and "the datacenter team" never does. No one is happy with that, and I still get the calls/email/messages, demanding the 'fix' or ETA. I would rather things be more under my control, because I inevitably am still 'blamed' when things go wrong.

Re: Ask HN: Do you self-host your database?

#124

Post by bsenftner »

bkovacev wrote: post #97

Where does one start when wanting to self-host a DB?

Look up the various posts praising SqlLite3. It is super easy to use. So simple I jokingly suspect conspiracy for why it is not the dominate DB solution everywhere.

I work in C++ and have made a super-butt simple wrapper for SqlLite3, available as an open source lib: https://github.com/bsenftner/kvs

Re: Ask HN: Do you self-host your database?

#125

Post by andix »

mekster wrote: post #120
andix wrote: post #112

Earlier quoted context omitted.

The great benefit of docker is, that you can replicate the exact same environment on a number of different machines. So the developer on Windows has exactly the same database version and configuration as the production server. Especially useful if you use some extensions or external modules for your database. If you host something serious and you don't need special builds of databases it is probably better not to use…

I always thought using a local machine is an anti pattern. There are plenty of virtualization software, one should just run the same OS as the production via VMware locally or give each person their own dev environment in the cloud with the same OS and be done with it than trying to align the environment partially.

Doesn't docker do that for developers? You run a container and then applications run in a defined environment.

I don't know how you wan to enforce that all developers and all production machines use the exact same environment. Maybe your production still runs RHEL 7, so all the developers need to run their desktop on CentOS 7?

Or take open source projects. Isn't it easier to put in the readme: "To start developing install docker and run 'docker-compose up'" vs. "Please install Ubuntu 18.04 into a VM, those 100 packages, but make sure this 50 packages are not installed."

In my experience virtualized desktop environments are very painful to work on, even with fast connections.

Docker (or containerization in general) is very useful for development. Much more useful than for deployment/production.

Re: Ask HN: Do you self-host your database?

#127

Post by [deleted] »

mosselman wrote: post #116
simonw wrote: post #54

Earlier quoted context omitted.

I'd be fascinated to hear more about this. Where do the SQLite files live? Are you using some kind of NFS or EFS or similar for them? Sharding them across many machines? How are you handling backups and high availability? Are you using Litestream? How many SQLite files do your application servers have open at any one time?

Interested in more info as well. Also, how do you handle migrations, both schema as well as data?

[deleted]

Re: Ask HN: Do you self-host your database?

#128

Post by littlestymaar »

codewithcheese wrote: post #86
ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

Slack used to partition their databases per workspace (customer) and then moved away from it. The "Disadvantages" section lists reasons not to do that. https://slack.engineering/scaling-datastores-at-slack-with-v... Two disadvantages that stand out for me are: 1. You must be able to scale the database up to your biggest customer and down to the your smallest this can be increasingly difficult to provision. 2. You are…

> 1. You must be able to scale the database up to your biggest customer and down to the your smallest this can be increasingly difficult to provision.

IIRC, Slack had a lot of scaling issues when they won a deal with IBM. Jumping from medium-size customers to such a behemoth will obviously be a challenge, but a lot of the difficulties they faced was non-technical and more business-related (“What do you mean, you want our product but not our monthly plan charged per seat?”). Anyway, that's a nice problem to have for a start-up ;).

Re: Ask HN: Do you self-host your database?

#129

Post by DataBrokerLol »

teddyh wrote: post #119

“Self-host” is such a weird word. Having your own stuff yourself should be the default , should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job. I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then…

Can you provide some more information about the claim that the US has abysmal internet speeds and capacity?

Edit: Grammar.

Re: Ask HN: Do you self-host your database?

#130

Post by Labo333 »

ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

I'm very curious, how do you handle concurrency? Or is this why you have one db per client?
Post reply on HN