This is soooo awesome. I started rewriting SageMathCloud to use RethinkDB when I learned in May about your plans to support high availability. I've been rewriting everything, doing tests (building from sources, then using the beta you kindly provided), and finally after months of work, I'm ready to release the new version of SageMathCloud last night, but RethinkDB 2.1 isn't out yet. So I'm torn about whether to go with 2.1beta and cross my fingers, or just wait, or what. And this! Thank you so much. RethinkDB is, for my use, the first database I've ever actually really loved (and React.js+flux the first web framework). Here's my client code in case anybody is curious: https://github.com/sagemathinc/smc/blob/rethinkdb/salvus/ret...
RethinkDB 2.1 is out: high availability
21–30 of 108 posts
Re: RethinkDB 2.1 is out: high availability
#22Slava @ RethinkDB here. I'll be around all day to answer questions about the release (along with a few other engineers on our team). We're very excited about this release -- it makes the lives of RethinkDB users dramatically better because they won't have to wake up anymore in the middle of the night in case of most hardware failures :) It also took over a year to build and test, and has been one of the most challeng…
Have you Jepsen-tested the new 2.1 release? If so, what were its results?
Re: RethinkDB 2.1 is out: high availability
#23Earlier quoted context omitted.
I'm evaluating a move from CouchDB to another system purely as an easily sharded/replicated KV document store. Do you recommend Rethink for that specific use-case or is it overkill? Your docs seem to convey that this isn't really in Rethink's wheelhouse (in that RDB is more suited for querying or real-time updating).
If you like the general style of CouchDB but want scale and availability that's been proven in mission critical user facing deployments for years, we just did a blog post about moving to Couchbase http://blog.couchbase.com/2015/august/moving-couch
That's funny, I just installed that over the weekend and was testing it out. It's really good but what sort of through me off was the high $/node licensing fee if you want the Enterprise edition. It basically starts at $5k/year/node and that's just too oppressive.
Do you use the Community Edition and if so is it generally stable / suits your needs? I didn't read your article but I'll check it out tonight.
Re: RethinkDB 2.1 is out: high availability
#24Earlier quoted context omitted.
RethinkDB can definitely act as a distributed key-value store (and a pretty good one). But I think the choice is highly dependent on your use-case and requirements. Could you go into a little more depth wrt your workload? (e.g. number of records, average record size, read/write ratio, expected ops/second, etc.)
Yea of course. We're basically talking about 10/90 write/read on JSON documents ~100kb in size with very low load, probably no more than 10-20 ops/second at peak. Records now are at 200,000 projected to grow 250k/year.
Thanks! I'd definitely give RethinkDB a try, I think you'd be pleasantly surprised at how nice it can be for this. You'll also find that you might want to run ad-hoc queries on JSON documents (e.g. for analytics/exploratory analysis, etc.) and having ReQL at your disposal will be great for that.
IMO picking a KV store makes the most sense if you have insane performance requirements (e.g. millions of ops/second) where you need to squeeze every last drop of performance and the query layer gets in the way.
(Also if you have any questions and don't want to share more info publicly, feel free to shoot me an email -- slava@rethinkdb.com)
Re: RethinkDB 2.1 is out: high availability
#25Slava @ RethinkDB here. I'll be around all day to answer questions about the release (along with a few other engineers on our team). We're very excited about this release -- it makes the lives of RethinkDB users dramatically better because they won't have to wake up anymore in the middle of the night in case of most hardware failures :) It also took over a year to build and test, and has been one of the most challeng…
We're seeing serious memory leaks in 2.0.4~trusty.
http://glui.me/?i=uu1gqb3son0sbnn/2015-08-11_at_11.33_AM.png...
You can see where the memory sets itself during the rethink process restart, and then it slowly climbs back up to ~3gb, despite no new data being written to the system (all our RethinkDB data is generated during deploy time).
Any pointers? I'd be happy to provide any diagnostics.
Re: RethinkDB 2.1 is out: high availability
#26Slava @ RethinkDB here. I'll be around all day to answer questions about the release (along with a few other engineers on our team). We're very excited about this release -- it makes the lives of RethinkDB users dramatically better because they won't have to wake up anymore in the middle of the night in case of most hardware failures :) It also took over a year to build and test, and has been one of the most challeng…
Have you Jepsen-tested the new 2.1 release? If so, what were its results?
Yes. We did months of internal tests, and 2.1 passes Jepsen tests. We'd love for Kyle to do his own analysis once he gets some free time. In the meantime there is a bit more info on this in the blog post under "testing" headline.
Re: RethinkDB 2.1 is out: high availability
#27Slava @ RethinkDB here. I'll be around all day to answer questions about the release (along with a few other engineers on our team). We're very excited about this release -- it makes the lives of RethinkDB users dramatically better because they won't have to wake up anymore in the middle of the night in case of most hardware failures :) It also took over a year to build and test, and has been one of the most challeng…
We're seeing serious memory leaks in 2.0.4~trusty. http://glui.me/?i=uu1gqb3son0sbnn/2015-08-11_at_11.33_AM.png... You can see where the memory sets itself during the rethink process restart, and then it slowly climbs back up to ~3gb, despite no new data being written to the system (all our RethinkDB data is generated during deploy time). Any pointers? I'd be happy to provide any diagnostics.
You should open an issue on GitHub, so we can track it down: https://github.com/rethinkdb/rethinkdb/issues
It helps if you include information on your setup in the issue (for a list of helpful details, read "How to submit a bug report" here: http://rethinkdb.com/docs/crashes/).
Re: RethinkDB 2.1 is out: high availability
#28Slava @ RethinkDB here. I'll be around all day to answer questions about the release (along with a few other engineers on our team). We're very excited about this release -- it makes the lives of RethinkDB users dramatically better because they won't have to wake up anymore in the middle of the night in case of most hardware failures :) It also took over a year to build and test, and has been one of the most challeng…
We're seeing serious memory leaks in 2.0.4~trusty. http://glui.me/?i=uu1gqb3son0sbnn/2015-08-11_at_11.33_AM.png... You can see where the memory sets itself during the rethink process restart, and then it slowly climbs back up to ~3gb, despite no new data being written to the system (all our RethinkDB data is generated during deploy time). Any pointers? I'd be happy to provide any diagnostics.
Thanks for reporting this!
The image doesn't seem to load for me. Would you mind opening a GitHub issue (https://github.com/rethinkdb/rethinkdb/issues/new) with as many details as possible? We need to know a bit more about the infrastructure/workload to replicate this; once we get the info we'll get to the bottom of it ASAP.
Also feel free to e-mail me at slava@rethinkdb.com if you need urgent/critical support.
Re: RethinkDB 2.1 is out: high availability
#29Slava @ RethinkDB here. I'll be around all day to answer questions about the release (along with a few other engineers on our team). We're very excited about this release -- it makes the lives of RethinkDB users dramatically better because they won't have to wake up anymore in the middle of the night in case of most hardware failures :) It also took over a year to build and test, and has been one of the most challeng…
I'm excited to work with RethinkDB on my pet projects. Glad to hear you guys beefed up failovers, that's a big deal in a lot of places. ;)
It's a joy to use for side projects! I'm using it as the datastore in a web app for learning that automatically generates quiz questions from user-provided data. Integrating with it using the Python API has thus far been dead simple.
Re: RethinkDB 2.1 is out: high availability
#30Earlier quoted context omitted.
I'm curious about "We chose to only store cluster metadata in Raft due to performance limitations imposed by distributed consensus" I would naively assume that since you need to replicate your database to the slave servers, you need to send the WAL or changes somehow anyway. Could you elaborate on what was the problem there? I'd be surprised if it is because of a fundamental limitation of Raft (but could easily be a…
Suppose you configure RethinkDB to store three copies of the data. When you do a write, RethinkDB will replicate the data to three nodes; that operation is roughly a matter of sending three messages to the appropriate nodes, getting the acks, and sending the ack back to the client. If we used Raft to replicate the document, in many cases it would be a lot more chatty. If you look at the Raft paper and track all the m…
OK, so it's a latency concern, which makes perfect sense.
> If we used Raft to replicate the document, in many cases it would be a lot more chatty.
I'm out of my depth here. Leader needs to send AppendEntries and slave needs to apply to persistent storage and ACK. Leader needs to wait for majority of ACKS before responding to the client. That's the same as your three-node replication scenario, so what am I missing here? Does Rethink db relax consistency guarantees in some cases to achieve better latency?
It's not your job to educate me on Raft, I appreciate your being patient with me, but feel free to opt out of this conversation anytime you please.