Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 236 posts

Re: Ask HN: Do you self-host your database?

#191

Post by coopsmoss »

howaboutnope wrote: post #163

The idea of "outsourcing blame" mentioned in several comments seems really weird to me: If I make the choice to outsource something, and whoever I outsourced it to fucks up, I'm still the one who made the decision to outsource it. The same goes for outsourcing to someone who does a better job than I could have: then that was a great idea, and yay for me.

It's always good to have a scapegoat. A lot of Open Source companies that sell support services are in many cases really selling themselves as a blameable party.

Re: Ask HN: Do you self-host your database?

#193

Post by vira28 »

bognition wrote: post #159

The company I work for self hosts our databases. We self host HBase, Elastsicsearch, MySql w/ Vitess, Kafka, and a few other things. Our scale is large enough so it makes a lot more sense to pay for engineers rather than to pay for a service. I work on the Data Infrastructure group and am more than happy to answer any questions about it

Thanks. How do you handle a. TLS certificates? b. Anything other than firewall restrictions for DB servers?

Re: Ask HN: Do you self-host your database?

#194

Post by liveoneggs »

ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

https://www.actordb.com/docs-howitworks.html#overview

Re: Ask HN: Do you self-host your database?

#196

Post by zo1 »

Moto7451 wrote: post #160
Jnr wrote: post #117

Earlier quoted context omitted.

I used it 10 years ago and initially locks were a slight problem. Not sure if this has been solved somehow in sqlite itself. I solved the problem by handling all the DB operations in a separate thread that handled write and read queues. It certainly was not as easy as using mysql/postgresql where server does everything for you. Nowadays there are probably plenty of libraries that handle that for you.

This pattern is what I did when I used SQLite in production. I used a lightweight actor model microframework to make it easier to route write requests from multiple areas of the code to the single writer thread and offer a continuation (or async/await) perform any follow up actions. Performance easily went beyond the requirements for the app.

I'm not sure how to express my reaction to this kind of design/architecture without being rude. Especially with performant open source databases being available. Was this some sort embedded app or toy project?

Re: Ask HN: Do you self-host your database?

#197

Post by anonfornoreason »

hypertele-Xii wrote: post #131
eric4smith wrote: post #17

IMHO self hosting your database (even in the cloud) is the best way to do it. You have control over the version. You have control over features. You have control over performance. It’s tons cheaper for greater performance - especially when you go over a few hundred gigs. Yes, the hosted ones have built in replication - but my data is far too valuable to put in the hands of a third party. If they lost it - they could…

And every thing you have control over you are also responsible for maintaining. If that's your thing, ok, but not everyone wants to be a DBA all day.

self hosting a now 300gb database using mariadb, been in production for 14 years with almost no downtime and very good performance. i do not DBA all day, i do it maybe once a year to do a version upgrade.

this perception of having to be special or only focus on sysadmin stuff for self hosting is incredible. cloud providers have managed diamond cartel levels of consumer propaganda in the tech sphere, its pretty amazing once you notice it.

Re: Ask HN: Do you self-host your database?

#198

Post by anonfornoreason »

tester34 wrote: post #156
torgard wrote: post #146

Earlier quoted context omitted.

> initial setup and configuration? If you think there is nothing more to maintaining a database than initial setup and configuration, you are in for a rude awakening. Shit happens.

I mean how much time daily would it take on average? I felt like generally you have nothing to do, except sometimes you gotta do something like respond to disk quota alert and stuff

you dont need to do much, i would say my time budget on self hosted replicated and backed up MySQL database administration is under 20 hours per year. yes the first couple years took more time (learning, building out a backup strategy) but id have the same amount of learning with a hosted solution so i don't consider that part of time spent.

Re: Ask HN: Do you self-host your database?

#199

Post by bob1029 »

Yes, but we took it 1 step further and put the database inside the application.

Sounds crazy until you learn that you can expose user-defined functions to SQL and eliminate all network overhead when it runs in the same process. SQLite operations are effectively a direct method invocation across a DLL boundary. If you want queries to reliably resolve within microseconds, this is a great path to go down.

Not for every application, but it fits ours extremely well. Deploying our software to customer environments is trivial because we only have to worry about the 1 binary image. No external databases, docker hosts, etc. are required to be installed.

Re: Ask HN: Do you self-host your database?

#200

Post by txprog »

ciconia wrote: post #52

We're currently transitioning from a multi-tenant 2TB postgres DB hosted on AWS RDS to using sqlite instead, a separate database for each client. We're doing this for multiple reasons: a) As our DB grew the service became very expensive, one of the biggest items in our AWS invoice; b) Keeping the PG servers up to date is a pain, we simply don't have time for this; c) We wanted to be able to migrate to other clouds an…

Very interested to hear your story and details!
Post reply on HN