The reason: ha for databases is hard.
However galera cluster and/or percona xtradb cluster work remarkably well, considering they’re open source.
Ask HN: Why are there no easy-to-use highly-available open source databases?
11–20 of 42 posts
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#12> What do I mean by Fake Open Source? A project that has a large percentage of its contributors beholden to a single organization/entity to me is not really open source in spirit. Then you should use a different term - something like "community project". The single organization projects are still open source, both technically and in spirit. > I'm looking for a project where I can feel confident my contributions won't…
I have contributed code to projects "owned" by one of these types of organizations and after 8 months of waiting for my (very small) patches to be reviewed because I wasn't internal, it just didn't feel like healthy Open Source. Calling them Fake Community Projects as a term works just as well for me.
OpenSearch could be a contender (though I did not have fun running Elasticsearch back in the day). I will keep watching it and see how development goes, it's a good call-out.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#13One thing which Percona has which MySQL and MariaDB does not is mature operators with HA support, which if you're using Kubernetes make High Availability much easier
https://www.percona.com/software/percona-kubernetes-operator...
Disclosure: I'm CEO at Percona
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#14I'm not sure if you're just picky or discerning but it seems you can find a reason to exclude anything if all you do is look for reasons to exclude. Why not just use SQLite with streaming replication? It should fit your bill. Databases rarely have what you would define as real open source with real contributors because the nature of a database means you need one owner and that owner has to be picky and exclude things…
I am very interested in any project that implements multi-active SQLite clusters with no cluster size limit, do you have any links for me?
rqlite is one of the ones that comes up in discussions. I can't offer a recommendation for or against though, as I've never needed to use it.
Not sure why "no cluster size limit" is important given the goals you listed in your comment btw. The rqlite faq mentions a practical cluster size limit of 9 read-write nodes (higher for read-only replicas).
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#15Earlier quoted context omitted.
I am very interested in any project that implements multi-active SQLite clusters with no cluster size limit, do you have any links for me?
rqlite is one of the ones that comes up in discussions. I can't offer a recommendation for or against though, as I've never needed to use it. Not sure why "no cluster size limit" is important given the goals you listed in your comment btw. The rqlite faq mentions a practical cluster size limit of 9 read-write nodes (higher for read-only replicas).
Rqlite is an awesome project but it's not intended for use above a couple GB, like etcd. Sorry, dataset size is what I meant by cluster-size which was a confusing way to put it. You're right, I don't care about the number of nodes in the cluster.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#16I'm not sure if you're just picky or discerning but it seems you can find a reason to exclude anything if all you do is look for reasons to exclude. Why not just use SQLite with streaming replication? It should fit your bill. Databases rarely have what you would define as real open source with real contributors because the nature of a database means you need one owner and that owner has to be picky and exclude things…
I am very interested in any project that implements multi-active SQLite clusters with no cluster size limit, do you have any links for me?
Just searching for SQLite replication will get you several options to research to fit your requirements. The concept of multi active and no size limit you would simply solve with a sharding mechanism that you layer on top through a data access method.
But if you're looking for some monolith project to provide all of this for you you're going to be in that territory of that fake open source. Because again to build all of this into one tool that has everything from the data access layer all the way down to the metal underneath to ensure that it's replicated and highly available is not a trivial task. Especially if you want things like acid compliance arbitrary joins between data structures and so forth.
The file system is effectively a database of things with an API that accesses those things. Simply a file system of properly named files and directories could meet your needs depending on what those are.
It's the complexity of your access model and methods that will determine what fits your bill.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#17Earlier quoted context omitted.
I am very interested in any project that implements multi-active SQLite clusters with no cluster size limit, do you have any links for me?
Just searching for SQLite replication will get you several options to research to fit your requirements. The concept of multi active and no size limit you would simply solve with a sharding mechanism that you layer on top through a data access method. But if you're looking for some monolith project to provide all of this for you you're going to be in that territory of that fake open source. Because again to build all…
Yeah, sounds like most folks agree this doesn't exist simply because it's very challenging to implement.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#18One thing which Percona has which MySQL and MariaDB does not is mature operators with HA support, which if you're using Kubernetes make High Availability much easier https://www.percona.com/software/percona-kubernetes-operator... Disclosure: I'm CEO at Percona
I agree, if you want to offload a lot of this to Kubernetes there seems to be quite a bit more out there. Percona has some neat stuff.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#19Earlier quoted context omitted.
rqlite is one of the ones that comes up in discussions. I can't offer a recommendation for or against though, as I've never needed to use it. Not sure why "no cluster size limit" is important given the goals you listed in your comment btw. The rqlite faq mentions a practical cluster size limit of 9 read-write nodes (higher for read-only replicas).
Rqlite is an awesome project but it's not intended for use above a couple GB, like etcd. Sorry, dataset size is what I meant by cluster-size which was a confusing way to put it. You're right, I don't care about the number of nodes in the cluster.
Ah, you're right, sorry. I naively assumed rqlite would cope with similar dataset sizes to sqlite itself, which I think is likely to be limited by how much disk you can put on a single node. But it seems rqlite is much more limited on that dimension, so it doesn't fit your criteria.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#20> YugabyteDB: Fake Open Source. Special shout out here for not even linking to instructions for how to build the database in the readme.
All the features are open source. Here is how to build from source https://docs.yugabyte.com/latest/contribute/core-database/bu....
> What do I mean by Fake Open Source? A project that has a large percentage of its contributors beholden to a single organization/entity to me is not really open source in spirit.
Well, somebody gotta start the project, no? Feel free to contribute though. Since it reuses PostgreSQL, it directly inherits the "postgresql community commits". The same with being a fork of Apache Kudu fork.
> If there's an "Enterprise" product and the organization calls the source code for the main project the "Community Edition" or something like it, it's not Real Open Source.
The "Enterprise" edition is "just" some scripts that make deployment & monitoring easier (and includes 24/7 developer support). All c++ features are open source.
And it's still young. You can't compare against PostgreSQL that has 20+ years of being available.