Live data from Hacker News

HyperDex: A searchable distributed key-value store

hyperdex.org

31–40 of 44 posts

Re: HyperDex: A searchable distributed key-value store

#31
post #26

Look interesting. Is like a mongodb + redis in one? If i understood it well, if the coordinator die then everything die? Any plan in provide pub/sub? And/or capped collections (by number (keep 1000) or TTL (keep 24h))

Great, let me answer your questions one by one:

HyperDex is a next generation NoSQL store, so it kind of resembles traditional NoSQL stores like Mongo, Cassandra, Riak, and Dynamo, with a rich interface similar to that of Redis, but it offers much stronger properties than all of these systems. It differs from Mongo in that it provides much stronger consistency and fault-tolerance guarantees. Mongo's default will gladly pretend that an operation was committed even before it was seen by any server. HyperDex differs from Redis in that it shards its data across a network and is generally designed from the ground up for a networked environment. Both Redis and Mongo can and will return stale results following a failure whereas HyperDex will always return consistent results -- it guarantees something called "linearizability" for key-based operations, which roughly means that time will never go backwards from the point of view of any client. And in spite of offering stronger properties, HyperDex is faster than both Mongo and Redis on industry-standard benchmarks.

The HyperDex coordinator sounds like a singular entity, but is in fact a redundant, replicated service. The Paxos algorithm (provided by the ConCoord implementation here: http://openreplica.org) ensures that the coordinator overall can survive failures of some of the coordinator replicas.

Building a pub/sub system on top of HyperDex would be an excellent project.

HyperDex does not support "capped collections" out of the box, but it would be trivial to implement these with a background thread that prunes the database. The data store supports sorted queries, so you can say "return the top-1000 objects sorted by insertion time" or whatever else metric you liked to sort by. And you can delete groups of objects. These operations are implemented efficiently.

Hope these help.

Re: HyperDex: A searchable distributed key-value store

#32
post #12

Earlier quoted context omitted.

CAP, as stated, is a tautology. If you want to understand this better, please read our description of the CAP theorem ( http://hyperdex.org/extras/ ). As formulated by Gilbert and Lynch, the CAP theorem really says, "If you must always read the latest value, and you must have any server serve any request, and clients are collocated with servers, then you cannot survive a partition." This is in the same class as, "If…

I was guilty, for a time, of flinging 'CAP' around. Matters are, I believe, both simpler and far more complex. Here's the simplest way I can imagine a scenario. This is very similar to what my company deals with all the time. Given two, network distant data centers. Requests can land on either datacenter. A request can be a PUT or a GET. We deal with phone calls, so we have to provide an exceptionally reliable and co…

HyperDex performs best in a single-datacenter environment. You are right that synchronous replication between datacenters can (and likely will) be slow.

There's much work on this (my favorites are Walter and COPS from SOSP 11) that is on-going. We'll likely be throwing our hat into the ring soon as well.

You are right that pushing some logic to the "thick client" is one of the viable solutions.

Feel free to contact us as you work on your PoC. We'd love to hear about positive results and help with any rough patches you encounter.

Re: HyperDex: A searchable distributed key-value store

#33
post #29
post #25

Earlier quoted context omitted.

f is the number of failures that the system can tolerate. Typical systems can tolerate f failures with either (f+1) or (2f + 1) replicas. HyperDex can tolerate f failures with f+1 replicas. This failure threshold is per region of the hyperspace. Within each region, servers are arranged in a chain. As servers fail they are removed from the chain. As new servers come into the cluster they are appended to the end of one…

"If failures are totally independent"...that's sometimes a big if, especially on cloud providers. I didn't see anything in the documentation about backups...what are the options?

Most of the decent providers will help you improve failure independence. I know first hand that Linode is very accommodating for this.

I mention independent failures because f=3 is sufficient for this scenario for most. If there is correlation between failures and you cannot move servers around then you can compensate with a higher f.

In a future release (within the next six months), we'll be adding support for consistent snapshots that guarantee that HyperDex can withstand more than f failures with a bounded amount of data loss.

It's always possible to retrieve all data with an empty search and manually dump it into another form.

Re: HyperDex: A searchable distributed key-value store

#34
post #33
post #29

Earlier quoted context omitted.

"If failures are totally independent"...that's sometimes a big if, especially on cloud providers. I didn't see anything in the documentation about backups...what are the options?

Most of the decent providers will help you improve failure independence. I know first hand that Linode is very accommodating for this. I mention independent failures because f=3 is sufficient for this scenario for most. If there is correlation between failures and you cannot move servers around then you can compensate with a higher f. In a future release (within the next six months), we'll be adding support for consi…

Cool. A related question...what if >f machines go down, but you can bring them back up? Do you lose data or just lose availability for a while?

Re: HyperDex: A searchable distributed key-value store

#36
post #35

How hard is it to add APIs for other languages? (In my case I'm thinking JVM, since I'm getting into Clojure.)

We have complete Java bindings (https://github.com/rescrv/HyperDex/commit/1c4e98d88517a63bf0...). Does this help for Clojure?

Our base bindings are in C. Any language which can wrap C can make bindings.

Re: HyperDex: A searchable distributed key-value store

#39
post #36
post #35

How hard is it to add APIs for other languages? (In my case I'm thinking JVM, since I'm getting into Clojure.)

We have complete Java bindings ( https://github.com/rescrv/HyperDex/commit/1c4e98d88517a63bf0... ). Does this help for Clojure? Our base bindings are in C. Any language which can wrap C can make bindings.

Awesome, yes.

Re: HyperDex: A searchable distributed key-value store

#40
post #25
post #24

Earlier quoted context omitted.

Thanks for explaining. Your project looks very interesting. I will be definitely following it. Can you explain some more about how node failure are handled? And whether there is a way to tune that. I.e more about the phrase "less than f nodes fail at any give point in time". Is that tunable and how dynamic is that. (Set once or dynamic setting? Also set per running cluster, per name space, per key-value pair?) Also,…

f is the number of failures that the system can tolerate. Typical systems can tolerate f failures with either (f+1) or (2f + 1) replicas. HyperDex can tolerate f failures with f+1 replicas. This failure threshold is per region of the hyperspace. Within each region, servers are arranged in a chain. As servers fail they are removed from the chain. As new servers come into the cluster they are appended to the end of one…

Thanks again for explaining and great work. Looks like a very exciting project!
Post reply on HN