21–30 of 35 posts
This is pretty exciting. A couple of questions: How are you (Tokutek) planning to keep up to date with the MongoDB tree? Are you planning on talking to the MongoDB folks about upstreaming this? Or will it be a pure fork with no sharing either way?
Since it doesn't support cross-shard atomic operations, I'm assuming it takes the same approach as RethinkDB, i.e. co-locating data with its index entries, which means index lookups require querying every shard in the cluster. But I don't see this explicitly stated in the user's guide, so it would be nice to get some confirmation.
I'll ask the same question that I had about RethinkDB last week [1]: how does TokuMX handle secondary indexes on a sharded database? Since it doesn't support cross-shard atomic operations, I'm assuming it takes the same approach as RethinkDB, i.e. co-locating data with its index entries, which means index lookups require querying every shard in the cluster. But I don't see this explicitly stated in the user's guide,…
For 1.0 we wanted to match mongodb feature-wise as broadly as possible.
However, fractal trees can handle multiple clustered indexes very well, and this extends to a few ideas I have for adding things to do sharding in what I think will be a much better way. It's a bit up in the air right now but I can tell you we plan to make some significant innovations here, and if you have experience you can share, you should email us and we can talk it over and maybe change what we end up doing.
I'll ask the same question that I had about RethinkDB last week [1]: how does TokuMX handle secondary indexes on a sharded database? Since it doesn't support cross-shard atomic operations, I'm assuming it takes the same approach as RethinkDB, i.e. co-locating data with its index entries, which means index lookups require querying every shard in the cluster. But I don't see this explicitly stated in the user's guide,…
Have you evaluated alternative compressors to QuickLZ? LZ4 and Snappy both offer comparable compression speed and much faster decompression, which would seem to be a big win for read-heavy workloads.
Have you evaluated alternative compressors to QuickLZ? LZ4 and Snappy both offer comparable compression speed and much faster decompression, which would seem to be a big win for read-heavy workloads.
I'll ask the same question that I had about RethinkDB last week [1]: how does TokuMX handle secondary indexes on a sharded database? Since it doesn't support cross-shard atomic operations, I'm assuming it takes the same approach as RethinkDB, i.e. co-locating data with its index entries, which means index lookups require querying every shard in the cluster. But I don't see this explicitly stated in the user's guide,…
These are only initial impressions, but:
* It's definitely faster. Our write behavior is largely upserts against integers and doubles, and I'm seeing roughly 100% improvement against stock Mongo 2.4. The machine in question is an m2.2xlarge with a 1000 piops EBS volume attached, and it's doing about 7000 update operations a second. ( safe mode )
* I'm seeing consistently lower IO util than stock Mongo. Stock tends to vary wildly between 200-750 write ops, while under sustained write traffic, I see about 250 write ops.
* CPU usage is pretty well balanced against all cores, as opposed to stock's behavior.
* It's too early to say whether or not the storage savings will be as good as claimed, but at this point it seems that the TokuMX reprs are about 40% of the stock reprs. Like I said, most of our data is ints and doubles tho.
VERY LARGE CAVEAT I'm not running the database in a replica set because I'm lazy. So, the write throughput numbers are likely the best case scenario of what you'd actually be running in production.
(edit: line spacing)
If you've got to use MongoDB, it seems pretty nice. If it came with a tiny person that maintained the database for you as well, it'd be a no brainer.