Ask HN: Why are there no easy-to-use highly-available open source databases?

I'm a single engineer who wants to run a database by myself, no cloud vendors involved. I don't really care about scalability to a point, I don't really care about the API to interact with the database. I just want something that I can set up on multiple machines with minimal effort and have the loss of one of those nodes not cause catastrophic failures. I want the experience of something like etcd without being limited to a few gigabytes of data in the cluster (a Terabyte or maybe a couple hundred Gigabytes as a limit would probably be fine). I've spent a lot of time looking through the various options and wrote up my thoughts on why each of them wasn't a good fit.

Postgres: A great database but HA options either have far too many moving parts, require manual intervention, are proprietary, or are Fake Open Source. Seems like there's lack of interest in addressing this because companies that sponsor development generate a lot of value off complex operations

MySQL: This database has some odd behavior but that would be fine if it weren't Fake Open Source

MariaDB: HA Options are poor or abandoned

SQL Server: Proprietary

Oracle: haha

CockroachDB: Fake Open Source

YugabyteDB: Fake Open Source. Special shout out here for not even linking to instructions for how to build the database in the readme.

MongoDB: Proprietary, and even before the license change it was Fake Open Source

Cassandra: Not fun to run, and even though I said I didn't care about API I don't necessarily love how it works. But it comes probably the closest?

ScyllaDB: Fake Open Source

TiDB: Fake Open Source

Singlestore: Fake Open Source

FoundationDB: This one comes close but its beginning as a proprietary database really hurt its community, which is way smaller now than it should be. Could grow into something great if more folks got behind it.

etcd: not suitable for use above a couple GB of data

What do I mean by Fake Open Source? A project that has a large percentage of its contributors beholden to a single organization/entity to me is not really open source in spirit. I'm looking for a project where I can feel confident my contributions won't effectively end up behind some proprietary license down the line if/when the VC backed organization that primarily sponsors development decides it needs to protect itself from AWS. If there's an "Enterprise" product and the organization calls the source code for the main project the "Community Edition" or something like it, it's not Real Open Source. If a single organization shuts down and contributions fall off a cliff (https://github.com/rethinkdb/rethinkdb/graphs/contributors) it's not Real Open Source. There are lots of Real Open Source projects with great communities of users/contributors, but many of the newer databases don't have legitimate open source development communities behind them, in my opinion.

I've probably missed some examples. Mostly, I wonder why there hasn't been a general purpose open source database that does the operations stuff as well as the proprietary databases do. Am I missing something?

Yep, your assessment seems broadly accurate to me. I was going to suggest Cassandra until I saw it in the list. I'm {interested in/optimistic about} FoundationDB too, although haven't had a chance to use it in practice yet.

Out of curiosity: what would your preferred choice(s) be to fit these requirements using existing proprietary products?

From my research I think Cockroach, Yugabyte, and Scylla are all interesting and solid products depending on the exact API/guarantees you're looking for. For products I don't even have the option to self-host I think DynamoDB and Cloud Spanner are great products, probably among others.

> What do I mean by Fake Open Source? A project that has a large percentage of its contributors beholden to a single organization/entity to me is not really open source in spirit.

Then you should use a different term - something like "community project". The single organization projects are still open source, both technically and in spirit.

> I'm looking for a project where I can feel confident my contributions won't effectively end up behind some proprietary license down the line if/when the VC backed organization that primarily sponsors development decides it needs to protect itself from AWS.

If you're talking about ElasticSearch, I'll point out that Amazon forked it and OpenSearch is not behind a proprietary license, so all contributions made to ElasticSearch continue to be available as open source, with improvements being made.

PostgresSQL - Not sure which HA solution you had experience with: https://patroni.readthedocs.io/en/latest isn't too bad. old adage comes to mind. Fast/Cheap/Good - pick 2 HA design are not all created equal. rubber stamp something HA often give false sense of security. HA to me is explicitly defined risk(down time) tolerance. For each of 9 it get more complex and cost goes up. Most commercial DB with HA are opinionated which is often the opposite ethos of open source