Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HyperDex: A Searchable Distributed Key-Value Store

hyperdex.org

Post reply on HN
Print view View original

71–80 of 89 posts

Re: HyperDex: A Searchable Distributed Key-Value Store

#71

Post by Craiggybear »

wickedchicken wrote: post #53
bickfordb wrote: post #27

Earlier quoted context omitted.

What's wrong with SQL?

* COBOL-inspired syntax that is neither human or machine parseable (easily). * Strange nested query rules * Language encourages Cartesian products for virtually every non-trivial calculation. This makes join order crucial for reasoning about performance. * Humans don't have the brain capacity to reason about these joins, forcing the logic onto the query planner. * Humans don't have the brain capacity to reason about…

What? SQL is virtually plain English and is efficient across a range of database architectures for moderate-to-complex join situations. Its just an implementation of Boolean Algebra with a thin abstraction layer that uses words like SELECT, JOIN, FROM, WHERE, etc.

If you have performance issues, its invariably because the structure of your data is less than optimal or you have many linked external tables. Or -- you are really bad at efficient query design. Even highly experienced SQL-ers always look at old queries and realise that they could be three times faster and half as big, and if there is a performance issue then that's the first thing they'll review.

Humans have been 'understanding' SQL very well for nearly forty years -- without modern refinements like graphical query designers, either.

Not all data is a suitable candidate for an RDBMS solution, I am the first to agree, but there's not actually much wrong with SQL when what you need is a relational table store.

There are a few bad implementations of it, though, it is true, and a lot of people who can't seem to grasp SQL for what it is.

Edit: I am myself a huge fan of NoSQL solutions and always seek to rationally justify a non-relational datastore solution where practical or possible. NoSQL isn't a replacement for SQL (with all its imperfections) for the kind of slicing and dicing where it is needed.

Re: HyperDex: A Searchable Distributed Key-Value Store

#72

Post by lobster_johnson »

ericmoritz wrote: post #4

From the FAQ " rel="nofollow">http://hyperdex.org/faq/> : "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? HyperDex is designed to operate within a single datacenter. The CAP Theorem holds only for asynchronous environments, and well-administered datacenters enable us to sidestep this tradeoff entirely." I'd like to see how they pull that off when a node goes down. I gue…

> "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing?

Which is wrong. You can only have two of the three at the same time: CA, CP or AP.

If they get something as fundamental as this wrong, you have to wonder about the rest of the project.

Re: HyperDex: A Searchable Distributed Key-Value Store

#73

Post by rescrv »

jfager wrote: post #28

I'm reading through the paper, and I'm curious if I understand the implications of key subspacing and value dependent chaining correctly - do all reads and writes for a given key get forced to a single node? I understand how the replication that's described allows for failover when the point-leader fails, but does it also allow for scaling writes and key lookups?

Reads and writes for a given key always go to a single node. As you add more machines with the same workload, you are handling fewer keys per machine. The flip side is, if the number of keys per machine stays constant, more machines directly translates into the ability to handle more keys.

HyperDex scales linearly as you can see in our scalability graph.

Re: HyperDex: A Searchable Distributed Key-Value Store

#74

Post by mononcqc »

lobster_johnson wrote: post #72
ericmoritz wrote: post #4

From the FAQ " rel="nofollow">http://hyperdex.org/faq/> : "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? HyperDex is designed to operate within a single datacenter. The CAP Theorem holds only for asynchronous environments, and well-administered datacenters enable us to sidestep this tradeoff entirely." I'd like to see how they pull that off when a node goes down. I gue…

> "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? Which is wrong. You can only have two of the three at the same time: CA, CP or AP. If they get something as fundamental as this wrong, you have to wonder about the rest of the project.

Technically you can have at most two of the three of the same time. It's well possible for some stores to provide only one or no property at all!

Re: HyperDex: A Searchable Distributed Key-Value Store

#75

Post by lobster_johnson »

mononcqc wrote: post #74
lobster_johnson wrote: post #72

Earlier quoted context omitted.

> "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? Which is wrong. You can only have two of the three at the same time: CA, CP or AP. If they get something as fundamental as this wrong, you have to wonder about the rest of the project.

Technically you can have at most two of the three of the same time. It's well possible for some stores to provide only one or no property at all!

Well, yes. But that's not what the FAQ says; it says you can have at most one. The distinction is significant.

Re: HyperDex: A Searchable Distributed Key-Value Store

#76

Post by ericmoritz »

j_baker wrote: post #48
nirvana wrote: post #42

If you're building a distributed KV stores, you should benchmark against other distributed KV stores. Mongo and Cassandra aren't really. But Riak is. Plus, since its "distributed" here's the benchmark I'd like to see: 1. Set up a cluster of 8 nodes. Set data replication to 3. 2. Load 3TB of data into the cluster, across 1M documents (or some data set of that order) 3. Run your tests. Optimize each of the DBs for the…

What exactly makes Cassandra "not really" a KV store, while Riak "really" is a KV store? I'm calling No True Scotsman.

In my opinion they're both KV stores. Where they differ is what data structure they store. Cassandra stores BigTable rows indexed by a key and Riak is a binary blob + metadata indexed by a key.

I suppose it all depends on how be define a key/value store. Is it simply a data structure that can be accessed by a primary key. If that's the fact, nearly all databases fall under that description. If, on the other hand, a key/value store is simply a database that stores a blob indexed by a key, then nirvana is correct in saying "Cassandra" is not a KV store, but where does that get us?

I think that comparing HyperDex to Cassandra is a valid comparison. Both support pk lookups and secondary key lookups. The same is true about Riak. It would be nice to see a YCSB benchmark for properly configured Riak cluster.

I am pretty ignorant of the YCSB benchmark but it would be nice to have a benchmark that tests fault tolerance as well as raw performance.

Re: HyperDex: A Searchable Distributed Key-Value Store

#77

Post by jfager »

rescrv wrote: post #73
jfager wrote: post #28

I'm reading through the paper, and I'm curious if I understand the implications of key subspacing and value dependent chaining correctly - do all reads and writes for a given key get forced to a single node? I understand how the replication that's described allows for failover when the point-leader fails, but does it also allow for scaling writes and key lookups?

Reads and writes for a given key always go to a single node. As you add more machines with the same workload, you are handling fewer keys per machine. The flip side is, if the number of keys per machine stays constant, more machines directly translates into the ability to handle more keys. HyperDex scales linearly as you can see in our scalability graph.

I think you're missing the point of the question. I'm not asking how you scale in general, I'm asking whether your replication scheme is only for failover or if it contributes to your scaling story - i.e., if a given key goes hot, does everything have to run through the point leader, or can each replica take writes/serve requests? It sounds like the former, which is fine, just wanted to clarify.

Re: HyperDex: A Searchable Distributed Key-Value Store

#78

Post by reitblatt »

lobster_johnson wrote: post #72
ericmoritz wrote: post #4

From the FAQ " rel="nofollow">http://hyperdex.org/faq/> : "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? HyperDex is designed to operate within a single datacenter. The CAP Theorem holds only for asynchronous environments, and well-administered datacenters enable us to sidestep this tradeoff entirely." I'd like to see how they pull that off when a node goes down. I gue…

> "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? Which is wrong. You can only have two of the three at the same time: CA, CP or AP. If they get something as fundamental as this wrong, you have to wonder about the rest of the project.

Or it could just be a typo on the brand new webpage of a brand new project. Which it obviously is if you look at their explanation of CAP.

Re: HyperDex: A Searchable Distributed Key-Value Store

#79

Post by lobster_johnson »

otterley wrote: post #68
rescrv wrote: post #50

Earlier quoted context omitted.

We use many lock-free datastructures which rely heavily upon the x86_64 architecture. Further, the expanded virtual address space enables us to mmap everything. All network traffic is packed, and in network byte order.

> the expanded virtual address space enables us to mmap everything This sounds like a disaster waiting to happen when a node's working set is larger than RAM. Have you considered the impact on performance due to excessive I/O resulting from this kind of overcommit? Redis performance, for example, suffers tremendously when its database size exceeds the available RAM, which is why the authors advise implementors to cap…

> This sounds like a disaster waiting to happen

Well, define "disaster". A memory-mapped dataset will cause pathological performance only when there is thrashing: If you are accessing a small percentage of the entire dataset, then unused data will be paged out and remain paged out. If the bulk of data being accessed exceeds the amount of available physical RAM, then you will get I/O trashing.

Memory-mapping is a good alternative to static allocation or a home-grown paging system because it lets the kernel handle the dynamics of allocation, letting your application transparently and gracefully handle RAM tension situation by relinquishing memory space to other apps. Kernels (including Linux and Windows) and CPUs are extremely efficient at paging I/O, much more efficient than a hand-written paging system because there's no need for the application's code to check whether a page is in physical memory — that's handled by the CPU itself.

Of course, any I/O incurred by a too-large dataset will drastically reduce performance compared to in-memory speed. But paging in itself does not necessarily lead to "disaster".

Re: HyperDex: A Searchable Distributed Key-Value Store

#80

Post by rescrv »

lobster_johnson wrote: post #72
ericmoritz wrote: post #4

From the FAQ " rel="nofollow">http://hyperdex.org/faq/> : "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? HyperDex is designed to operate within a single datacenter. The CAP Theorem holds only for asynchronous environments, and well-administered datacenters enable us to sidestep this tradeoff entirely." I'd like to see how they pull that off when a node goes down. I gue…

> "So, the CAP Theorem says that you can only have one of C, A, and P. Which are you sacrificing? Which is wrong. You can only have two of the three at the same time: CA, CP or AP. If they get something as fundamental as this wrong, you have to wonder about the rest of the project.

Just a simple typo. I've fixed it.

Thanks for pointing this out.

Post reply on HN