41–50 of 89 posts
How does it compare to elasticsearch?
Elasticsearch is a Fulltext search solution not a K/V Database.
Plus, since its "distributed" here's the benchmark I'd like to see:
1. Set up a cluster of 8 nodes. Set data replication to 3. 2. Load 3TB of data into the cluster, across 1M documents (or some data set of that order) 3. Run your tests. Optimize each of the DBs for the best way to access them (Eg: link walking vs. map reduce on Riak if that's faster, or secondary indexes if that's faster, or Riak Search if that's faster-- there are many ways to search Riak.) 4. Throw out the results of #3. 5. Pull the plug on 2 nodes. EG: Shut down completely, no longer on the net work at all. Poof, gone. Pick the two nodes by rolling the dice. If you have SPFs, and a bad role of the dice would have brought the whole cluster down, remove the word "distributed" from your marketing. 6. Let the cluster sit for 30 minutes. 7. Run your benchmarks.
The benchmarks developed in #7 are the ones I want to see.
I have no clue how HyperDex would perform in this situation. It could kick Riak's butt (but then, small cluster performance is just one of the criteria that is important to me). I just wish people did benchmarks like this (though I know its a PITA to do it this way.)
Interesting.. keep it coming. We need a unified nosql language.. Basically what SQL is.
there is sparql
Earlier quoted context omitted.
Elasticsearch is a Fulltext search solution not a K/V Database.
Technically yes, but you can use elasticsearch as a distributed key-value store. I use it in production and it works amazingly well for me. Yes, searches may return stale data (one second old by default) but that is not much of a problem in my app.
EDIT: Color me surprised. I didn't see it mentioned at all in the ElasticSearch docs or in their Github repo, but apparently this is a fairly popular thing to do.
"Elastic search as a database?" http://stackoverflow.com/questions/6636508/elastic-search-as...
"ElasticSearch, datastore for user and social data?" http://stackoverflow.com/questions/8190288/elasticsearch-dat...
"Elasticsearch and NoSql database" http://stackoverflow.com/questions/8026880/elasticsearch-and...
From the FAQ " rel="nofollow">http://hyperdex.org/faq/> : "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? HyperDex is designed to operate within a single datacenter. The CAP Theorem holds only for asynchronous environments, and well-administered datacenters enable us to sidestep this tradeoff entirely." I'd like to see how they pull that off when a node goes down. I gue…
It's a bit of a middle ground. Yes, the replication is synchronous, which impacts availability. However, the master can remove a failed replica from the chain fairly quickly. In principle, with proper tuning, a node failure would merely cause a brief hiccup. This would feel more like a period of increased latency than a full-blown outage. So there really needn't be much sacrifice of availability. However, there's als…
Keeping consistency at the expense of availability in the event of a partition is precisely what makes this a CP system.
Earlier quoted context omitted.
This sounds like a CP system to me. There's nothing wrong with that btw, I don't know why people are so reluctant to admit this. AP systems have some useful properties, but they're also (typically) more difficult to reason about. The "hiccups" you describe are periods of unavailability. The increased latency is caused by an element of the system waiting for the data to become available again, a totally valid strategy…
To be precise, I would argue this is actually a CA system. "C" because there are consistency guarantees, which are upheld even in the face of failures / partition. "A" because the system will continue making progress even after a node failure. What I called a "hiccup" can be made arbitrarily short, in principle at least. The system can work around failed nodes, it does not need to wait for them to be repaired. Not "P…
If you're building a distributed KV stores, you should benchmark against other distributed KV stores. Mongo and Cassandra aren't really. But Riak is. Plus, since its "distributed" here's the benchmark I'd like to see: 1. Set up a cluster of 8 nodes. Set data replication to 3. 2. Load 3TB of data into the cluster, across 1M documents (or some data set of that order) 3. Run your tests. Optimize each of the DBs for the…
That's the point of YCSB. Each vendor can submit the optimal configuration for their system, and they all run the same benchmark.
At the end of the day, each vendor is going to publish benchmarks that show their system performing better than all others.
It's your job, not theirs, to verify those benchmarks for your particular work load.
Edit: Also, while Mongo wouldn't classify as a distributed store, Cassandra definitely would. In fact, it's more "distributed" than Riak, since you have to pay Basho for multiple DC support (unless that's in the open source version?), whereas Cassandra is completely free.
If you're building a distributed KV stores, you should benchmark against other distributed KV stores. Mongo and Cassandra aren't really. But Riak is. Plus, since its "distributed" here's the benchmark I'd like to see: 1. Set up a cluster of 8 nodes. Set data replication to 3. 2. Load 3TB of data into the cluster, across 1M documents (or some data set of that order) 3. Run your tests. Optimize each of the DBs for the…
Earlier quoted context omitted.
HyperDex uses value-dependent chaining, which offers fault tolerance properties similar to those provided by chain replication ( http://www.cs.cornell.edu/home/rvr/papers/osdi04.pdf ). A single node failure will be recovered from quickly without issue. Multiple concurrent failures are handled the same as the single failure case, so long as our failure assumptions are not violated (e.g., every node in the datacenter f…
That can work if the server process is killed so that the master is immediately notified. But what about other failure modes? For example, if the disk has soft errors and writes start taking several seconds to complete, the system can't decide in a small amount of time that the node is dead.
It must be able to do so, otherwise it will be deemed faulty by the entity it reports to.
the FAQ mentions that you target x86_64 architecture. Does the server work on i386? also, I hope it's doing proper memory alignment and endiannes independence? (because Mongo sucks on that)
All network traffic is packed, and in network byte order.