Redis, from the Ground Up • Hacker News Board

Re: Redis, from the Ground Up

#11

Post by silentbicycle » Mon, Oct 18, 2010, 10:28 PM UTC

antirez wrote: post #10

silentbicycle wrote: post #9

Earlier quoted context omitted.

Some people learn set theory before data structures and algorithms. People focusing on math rather than computer science, for example. As long as we're speculating about hypothetical aliens, I wouldn't reject that possibility. Arrays, linked lists, binary trees and mergesort could be more fundamental than the relational model, but the relational model in turn may be simpler than (say) red-black trees, BSPs, skiplists…

Well in a limited fashion this experiment could be actually be done. I'm not sure how this would be modeled, but I guess giving smart young guys that don't know nothing about programming a set of "tools" in a simulation that can be used to create the building blocks of many different data structures. Then provide problems, and look at what they do and invent in order to solve such problems. Will be very hard to do th…

Absolutely. Hence, thought experiment and aliens.

Having programmers explore in a (to them) really weird language (like Prolog or Haskell) and seeing what they come up with might be a good approximation, though.

I may never have thought of difference lists, but then Prolog made them seem obvious. :)

Re: Redis, from the Ground Up

#12

Post by joe_the_user » Mon, Oct 18, 2010, 11:41 PM UTC

Redis' internal design typically trades off memory for speed. For some workloads, there can be an order of magnitude difference between the raw number of bytes handed off to Redis to store, and the amount of memory that Redis uses.

What are the circumstances that make this kind of tradeoff worthwhile?

A generic Key-Value store, say Kyoto Cabinet, is pretty fast and you can configure its cache to be huge if you need it. Does reconstructing and using a list/set/hash take that much time?

Edit: Is the "order of magnitude" here greater or less than the extra space that keeping a b-tree index in memory would take? Is it doing something akin to that or a completely different thing?

Re: Redis, from the Ground Up

#13

Post by antirez » Tue, Oct 19, 2010, 8:41 AM UTC

joe_the_user wrote: post #12

Redis' internal design typically trades off memory for speed. For some workloads, there can be an order of magnitude difference between the raw number of bytes handed off to Redis to store, and the amount of memory that Redis uses. What are the circumstances that make this kind of tradeoff worthwhile? A generic Key-Value store, say Kyoto Cabinet, is pretty fast and you can configure its cache to be huge if you need i…

The tradeoff is especially worthwhile because we export complex data structures (I wrote a great deal of articles about this, please check the latest at antirez.com), but this time I'll try to provide a proof by paradox: what you are saying here is that memcached may be replaced with TC from the point of view of performances, if you add an LRU expiry, that I think it's not true.

Re: Redis, from the Ground Up

#14

Post by joe_the_user » Tue, Oct 19, 2010, 7:04 PM UTC

antirez wrote: post #13

joe_the_user wrote: post #12

Redis' internal design typically trades off memory for speed. For some workloads, there can be an order of magnitude difference between the raw number of bytes handed off to Redis to store, and the amount of memory that Redis uses. What are the circumstances that make this kind of tradeoff worthwhile? A generic Key-Value store, say Kyoto Cabinet, is pretty fast and you can configure its cache to be huge if you need i…

The tradeoff is especially worthwhile because we export complex data structures (I wrote a great deal of articles about this, please check the latest at antirez.com), but this time I'll try to provide a proof by paradox: what you are saying here is that memcached may be replaced with TC from the point of view of performances, if you add an LRU expiry, that I think it's not true.

I hope this didn't come off as a criticism. I was simply trying to understand what Redis does.

I'm using Kyoto cabinet to serialize various hashlist-based-classes and I'm wondering what bottlenecks etc. I might encounter are. I'll take a look at your site.