[deleted]
Ask HN: Why are there no easy-to-use highly-available open source databases?
21–30 of 42 posts
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#22You are missing something.
You list several completely different types of databases and then mention you are looking for a "general purpose database".
This tells me you do not understand the problem you are trying to solve, therefore do not know how to define the requirements nor features you require to solve it.
It also tells me you do not understand the difference between different types of databases, nor the niche they fill, let alone why their HA functions very differently.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#23> What do I mean by Fake Open Source? A project that has a large percentage of its contributors beholden to a single organization/entity to me is not really open source in spirit. Then you should use a different term - something like "community project". The single organization projects are still open source, both technically and in spirit. > I'm looking for a project where I can feel confident my contributions won't…
I have contributed code to projects "owned" by one of these types of organizations and after 8 months of waiting for my (very small) patches to be reviewed because I wasn't internal, it just didn't feel like healthy Open Source. Calling them Fake Community Projects as a term works just as well for me. OpenSearch could be a contender (though I did not have fun running Elasticsearch back in the day). I will keep watchi…
I totally feel this but it’s very dependent on the company
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#24Earlier quoted context omitted.
I am very interested in any project that implements multi-active SQLite clusters with no cluster size limit, do you have any links for me?
rqlite is one of the ones that comes up in discussions. I can't offer a recommendation for or against though, as I've never needed to use it. Not sure why "no cluster size limit" is important given the goals you listed in your comment btw. The rqlite faq mentions a practical cluster size limit of 9 read-write nodes (higher for read-only replicas).
rqlite author here. Happy to answer any questions.
Noting the title of this topic, rqlite is specifically designed to be easy to use. I know of no other HA database that is simpler to operate. Installation is trivial, and clustering is easy. You interact with it via a HTTP API.
And you get all the power and quality of SQLite as the database engine.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#25rqlite https://github.com/rqlite/rqlite
I'm the creator of this project. While it's not going to work super well at very large datasets, it's explicitly designed to be trivial to deploy, and very easy to operate. You can get it up and running in seconds, and clustering seconds later. My practical experience with databases told me that operating the database is at least as important as performing queries with it. So I put a lot of work into easy clustering, clear diagnostics, and solid code.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#26Earlier quoted context omitted.
rqlite is one of the ones that comes up in discussions. I can't offer a recommendation for or against though, as I've never needed to use it. Not sure why "no cluster size limit" is important given the goals you listed in your comment btw. The rqlite faq mentions a practical cluster size limit of 9 read-write nodes (higher for read-only replicas).
rqlite author here. Happy to answer any questions. Noting the title of this topic, rqlite is specifically designed to be easy to use. I know of no other HA database that is simpler to operate. Installation is trivial, and clustering is easy. You interact with it via a HTTP API. And you get all the power and quality of SQLite as the database engine.
You can go larger than 9, quite a bit larger. I just suggested 9 as a useful size. But the 7.x series introduced much better support for Kubernetes, so it's much easier now to manage larger clusters.
But larger clusters mean a larger quorum, so larger isn't really better. It depends on what you want. If you want a distributed database where more nodes means more write performance, then rqlite may not be a good fit. rqlite is distributed for fault-tolerance and HA, not for performance.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#27Earlier quoted context omitted.
Rqlite is an awesome project but it's not intended for use above a couple GB, like etcd. Sorry, dataset size is what I meant by cluster-size which was a confusing way to put it. You're right, I don't care about the number of nodes in the cluster.
Ah, you're right, sorry. I naively assumed rqlite would cope with similar dataset sizes to sqlite itself, which I think is likely to be limited by how much disk you can put on a single node. But it seems rqlite is much more limited on that dimension, so it doesn't fit your criteria.
Full details on what the limits in rqlite may be, and why, are here: https://github.com/rqlite/rqlite/blob/master/DOC/PERFORMANCE...
That said, very large datasets (> 2GB) will work, if you've got the disks and memory to handle it. The link above explains how to do it.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#28I have deployed and run Cassandra myself, basically as you describe.
The first 'dev' Cassandra install was three nodes. I downloaded the .tar.gz, installed it, started each process in turn on each node with the required configuration.
That was 2012 and there is a chance that cluster is still running. It was low-volume in terms of data. TTL configured so it would never run out of disk. Never had any issues in particular. I used it for ~5 years before concluding work with that client.
The problem in that case was Cassandra proliferated in that small company, they didn't build any particular expertise beyond me, and in the end I was being pulled into discussions from different teams, different products, split across about 13 clusters. Wasn't much fun - but that wasn't the DB's fault.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#29Yep, your assessment seems broadly accurate to me. I was going to suggest Cassandra until I saw it in the list. I'm {interested in/optimistic about} FoundationDB too, although haven't had a chance to use it in practice yet. Out of curiosity: what would your preferred choice(s) be to fit these requirements using existing proprietary products?
From my research I think Cockroach, Yugabyte, and Scylla are all interesting and solid products depending on the exact API/guarantees you're looking for. For products I don't even have the option to self-host I think DynamoDB and Cloud Spanner are great products, probably among others.
Cool, thanks - yep, DynamoDB seems fairly reliable, and that's also good to hear another vote in support of Cloud Spanner.
Glad to have learned about rqlite from this thread as well.
Re: Ask HN: Why are there no easy-to-use highly-available open source databases?
#30The community is still slowly recovering from the collapse of the company but RethinkDB is worth considering in your analysis.