Please stop using json as a data model. I have no idea how to represent dates, or timestamps, or colors, or any other unsupported data type.
RethinkDB: An open-source distributed database built with love over three years
61–70 of 247 posts
Re: RethinkDB: An open-source distributed database built with love over three years
#62Are there any performance tips or information? This looks really cool
Performance analysis of complex systems is really tricky, and good benchmarks are even trickier. We'll be working through posting numbers, docs, tips, etc. over the next few months. Documenting this well is very hard work, so it'll take a little bit of time. In the meantime, you can always chat with us and we'll help you work through any issues you might run into.
Can I provide an intelligent sharding algorithm in place of a naive partition by primary key? I always get into trouble encoding things into primary keys eventually.
Re: RethinkDB: An open-source distributed database built with love over three years
#63Why would you use this over PostgreSQL, especially with pg's new json support?
JSON support in Postgres is currently limited to a validated plain text field, it doesn't let you efficiently query inside the json object.
Re: RethinkDB: An open-source distributed database built with love over three years
#64Earlier quoted context omitted.
> All queries are fully parallelized Does it means that every query touches all servers ? Or does it sends queries to only a subset of servers when possible ? (e.g. range queries on PK)
Joe Doliner - RethinkDB engineer here. > Does it means that every query touches all servers ? No. > Or does it sends queries to only a subset of servers when possible ? (e.g. range queries on PK) The query planner distributes the query between the nodes that actually contain the relevant data. Here are a few examples: In your example, a range get on the primary key, the query would touch one copy of each shard of the…
> In your example, a range get on the primary key, the query would touch one copy of each shard of the table.
But shouldn't it be fewer than "each shard"?
Let's say the range is 3 < PK < 7. If all PKs in that range only lives in 2 shards (out of a total of say 10 shards) then the query should only be run in those 2 shards, no? Or will all 10 shards still be touched by the query?
Re: RethinkDB: An open-source distributed database built with love over three years
#65Earlier quoted context omitted.
Thanks -- will do in the next few days.
Agreed. It would be very useful as a potential user. As an example, I like how Basho provides some comparisons[1] of Riak vs other popular options. http://docs.basho.com/riak/latest/references/appendices/comp...
We're already working on something in this direction.
alex @ rethinkdb
Re: RethinkDB: An open-source distributed database built with love over three years
#66Last I heard RethinkDB was a tail-append style engine for MySQL that was optimized for SSDs. Interesting to see a drastic pivot like this. Looks good, and good luck.
Re: RethinkDB: An open-source distributed database built with love over three years
#67Nice work! It seems that you are well aware of the tradeoffs that you are taking and communicating it openly in your documentation (and your choices seem to be very reasonable). I really like the tone of your communication – it seems essentially BS/koolaid free.
1. How much data can you put in one instance before seeing performance degradation? I know that you still working on good benchmarks – but do you have any ballpark figures?
2. How does replication work? Is it closer to row/document or statement based (or something completely different)? How fast is the replication?
3. What is your envisioned used of the replication? Are replicas supposed to serve read traffic, or their goal is to keep the data safe in case of a catastrophe?
4. Can you tell me something more about cluster configuration propagation? The Advanced FAQ answer doesn't get into much detail.
5. Am I correct to assume that you are using protocol buffers? What motivated your choice?
Re: RethinkDB: An open-source distributed database built with love over three years
#68I'd really like to know why they don't have a PHP library, considering it powers half the web. The key shouldn't be to promote someone elses own language when using infrastructure products but rather to support everyone in using their tool. Mongo supports everything under the sun, so should this.
Re: RethinkDB: An open-source distributed database built with love over three years
#69Suggestion: It would be great to have a page on your website that explains why RethinkDB is better than the other prevailing options. Right now I don't know why I'd want to invest time setting up yet another database.
Thanks -- will do in the next few days.
What's the elevator pitch? Maybe we can help you with those advantages if you can tell us right now.
Re: RethinkDB: An open-source distributed database built with love over three years
#70I'm hoping this'll be a viable replacement for MongoDB. (Sparse/Schema-free is incredibly useful for me, as is JSON-centric modeling) jedberg already asked for a compare/contrast, but let me provide some specifics I care about that you might be able to answer. 1. Is it fair to say that thanks to MVCC, running an aggregation or map-reduce job isn't going to lock the whole damn thing up like it does on MongoDB? 2. You'…
1. Yes -- that was the main motivation for MVCC. We wanted to allow people to use rethinkdb for analytics and map/reduce on top of the realtime system without dealing with having to replicate data into something else. 2. Short answer: we favor consistency (via master/slave under the hood). It allows for much easier API, much fewer issues in production, etc. The user experience is just better. If you're ok with out of…
I want to preface my comment: this is impressive work, congratulations on shipping, and this is what MongoDB should have been from the start.
In reality, most transactional database deployments are heavily skewed towards read workload, so reading from hot slaves is basically a requirement for master/slave databases. So, in most real world applications at scale, apps already deal with inconsistencies between slaves and the master and are making the "difficult" choice of dealing with CAP trade-offs. Asynchronous replication also creates a potential for difficult or impossible to recover from data loss in the sense that masters & slaves always have a continuous possibility for split-brain.
RethinkDB does not provide multi-shard transaction atomicity and/or isolation, which in my experience is the biggest difficulty thrown up in front of developers coming from single-node databases. I feel like the difficulty of dealing with inconsistencies across multiple versions of a single object is far more familiar as most developers have at least dealt with cache invalidation in some form. It's really having to ensure and deal with potentially out of order operations (inconsistency in the ACID sense) across a "graph" of data that's more insidious.