Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Do you self-host your database?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 236 posts

Re: Ask HN: Do you self-host your database?

#31

Post by LAC-Tech »

geocar wrote: post #23

Yes. I even go so far as to embed the database in my applications. In general, this is implemented as two main flows: - Data collected from "the edge": Web servers that serve ads, or receive form fill-outs for lead generation, that do nothing but record this information in a logfile - Configuration and Reporting pushed from "the hub": A central processing node (usually in a hot/warm configuration, but I've been known…

So you're using web apps that run sqlite? With some logic on a central server, which then updates all the web apps?

Re: Ask HN: Do you self-host your database?

#32

Post by vira28 »

kureikain wrote: post #27

I self hosted everything from Postgres to ElasticSearch. IMHO, managed solution ins't that much reliable than running your own with the right planning. Failover strategy of AWS RDS is just absurb. It's literally just promote other instance, switch DNS over. And I do see issue where the new master are behind a few transaction compare with the replica... Managed database is just to move the responsibility when the data…

Amazing. What are some advices that you would give for someone who is looking to self host?

Re: Ask HN: Do you self-host your database?

#33

Post by BozeWolf »

Right now we (team of 12, 1 devops engineer) use mongodb atlas. While mongodb/nosql itself really is not giving us any value and that I really wished that postgresql was picked for our relational data, I must admit that atlas really is a nice product.

Hosting is not only about speed and price. We use atlas (=hosted mongodb), because we get that nice dashboard with all those statistics and performance tuning hints. Also, if we screw something up, mongo engineers are available immediately to help us. That was nice when we broke production because of index creation on the foreground. (Which pymongo defaults to, even though mongodb itself defaultst to backgound creation). We consulted them for tuning a frequently running “slow” query and that helped.

In short: hosted solution for support and less maintenance. Backups are arranged. We can choose which version to run on and upgrades are also arranged.

Question: is there a comparable service for postgresql like mongodb atlas?

Re: Ask HN: Do you self-host your database?

#34

Post by buro9 »

devoutsalsa wrote: post #24
buro9 wrote: post #15

Yes. A Postgres which is currently only 1TB in size. It's read heavy, the write workload is very low in tens to spikes of low hundreds per second whilst the read is in thousands per second. The workload is stable during traffic spikes due to app caching and CDN caching (for guests). I do this for cost reasons, self hosting the database is so much cheaper than the managed options that for the same cost I can run the e…

Cheaper in terms of $$$. If you like managing a database, I'm sure it's a good deal!

It was very little effort to setup. Used puppet to define everything I wanted the infra to be, and have pretty much left it be.

It's only reachable from the app servers, backed up to tarsnap.

It's really not a lot to learn and hasn't been a chore to operate.

Re: Ask HN: Do you self-host your database?

#35

Post by dataflow »

eric4smith wrote: post #17

IMHO self hosting your database (even in the cloud) is the best way to do it. You have control over the version. You have control over features. You have control over performance. It’s tons cheaper for greater performance - especially when you go over a few hundred gigs. Yes, the hosted ones have built in replication - but my data is far too valuable to put in the hands of a third party. If they lost it - they could…

> self hosting your database (even in the cloud)

> my data is far too valuable to put in the hands of a third party

Confused, you self-host "in the cloud" without a third party involved? What does that mean? Are you not counting the party hosting your machine?

Re: Ask HN: Do you self-host your database?

#36

Post by vira28 »

BozeWolf wrote: post #33

Right now we (team of 12, 1 devops engineer) use mongodb atlas. While mongodb/nosql itself really is not giving us any value and that I really wished that postgresql was picked for our relational data, I must admit that atlas really is a nice product. Hosting is not only about speed and price. We use atlas (=hosted mongodb), because we get that nice dashboard with all those statistics and performance tuning hints. Al…

Makes sense.

> Question: is there a comparable service for postgresql like mongodb atlas?

There is no company behind Postgres. So, the big public cloud providers are providing their hosted solution.

Re: Ask HN: Do you self-host your database?

#37

Post by jesterson »

eric4smith wrote: post #17

IMHO self hosting your database (even in the cloud) is the best way to do it. You have control over the version. You have control over features. You have control over performance. It’s tons cheaper for greater performance - especially when you go over a few hundred gigs. Yes, the hosted ones have built in replication - but my data is far too valuable to put in the hands of a third party. If they lost it - they could…

and most importantly, you have control over your DATA.

Re: Ask HN: Do you self-host your database?

#38

Post by remus »

tpetry wrote: post #26

I do host a medium-sized PostgreSQL database, but i would love not to. I need the control you won't have with a cloud offering (you can't install extensions) but i don't want to do all the other steps, and you always have the fear that backups do not work correctly. And i am not the only one hosting it by my own, many people are using self-hosting PaaS like dokku, flynn or caprover. And all these solutions have a com…

> I need the control you won't have with a cloud offering (you can't install extensions)

Depends on who you go with. Google's managed postgres thing lets you install supported extensions.

https://cloud.google.com/sql/docs/postgres/extensions

Re: Ask HN: Do you self-host your database?

#39

Post by devoutsalsa »

buro9 wrote: post #34
devoutsalsa wrote: post #24

Earlier quoted context omitted.

Cheaper in terms of $$$. If you like managing a database, I'm sure it's a good deal!

It was very little effort to setup. Used puppet to define everything I wanted the infra to be, and have pretty much left it be. It's only reachable from the app servers, backed up to tarsnap. It's really not a lot to learn and hasn't been a chore to operate.

I'm mostly thinking about all the stuff that goes wrong. Corrupted data files, server(s) die, monitoring, getting paged, backups, testing restores, yada yada. It works until it doesn't, and then the value you derive from self-hosting is questioned when there's a failure. If you can self-insure against shit happening & like doing so, then it's probably worth it. Mostly just point this out for the potential self-hosters who haven't thought everything through, and more likely (like myself) who wouldn't know what to think of in the first place.
Post reply on HN