For a post that attempts to compare Redis to memcached for caching, it's amazing how few actual numbers appear in the post.
"Actual numbers" for caching services only have meaning in the context of a particular application.
Clarifications about Redis and Memcached
11–20 of 43 posts
Re: Clarifications about Redis and Memcached
#12If data access is equally distributed across keys, then running multiple Redis instances (with sharding) is perfect, since ultimately you want each key to be serviced by a single core, for optimal performance in a system with caches and NUMAs.
However, if data access is read-heavy and not equally distributed across keys, then having shared memory (aka multithreading) is quite essential since you want all cores to operate on the same data, and shared memory is much better than sending the data across especially if the heavily accessed keys vary quickly over time.
I'm not sure if memcached handles this well (it requires a lightweight rwlock/RCU/MVCC mechanism, for instance), but a shared-nothing system like Redis cannot provide good performance in all cases.
Re: Clarifications about Redis and Memcached
#13If data access is equally distributed across keys, then running multiple Redis instances (with sharding) is perfect, since ultimately you want each key to be serviced by a single core, for optimal performance in a system with caches and NUMAs. However, if data access is read-heavy and not equally distributed across keys, then having shared memory (aka multithreading) is quite essential since you want all cores to ope…
Good point, but I could add that case 1 is vanishingly unlikely to happen in a real system. You always have hot keys, i.e. a Zipfian distribution. That's almost tautological for a cache -- by using a small amount of space you can handle most of the reads.
So, not knowing much about Redis, I would conclude based on this blog post that memcached has a pretty big advantage as a cache in real systems (multithreading).
Re: Clarifications about Redis and Memcached
#14[deleted]
Re: Clarifications about Redis and Memcached
#15Both are relatively small projects and really there is no much stuff to compare.
I'm slowly working on my own memcached clone and plan to add persistence there eventually. Not so sure about replication, it might be too hard. In principle, after I'm done with getting a better version of memcached, both feature-wise and performance-wise, if time permits, I can also add the redis protocol support.
Therefore, it will be possible to have both in a single package and don't worry finding which one is better.
This is my project, if anyone is interested: https://github.com/ademakov/MainMemory
[Update: my statement was only about cache-related functionality, admittedly redis supports very interesting data structures, persistence, replication. But as just a very fast in-memory cache, there is nothing particularly advanced in either case. On the other hand there are projects like RAMCloud, Seastar that I find inspiring when I work on my own project.]
Re: Clarifications about Redis and Memcached
#16I dropped Memcached in favor of Redis for caching a long time ago, because as far as it matters for my purposes, Redis is a strictly-superior superset of Memcached's functionality, and I have no desire to maintain multiple pieces of software in the stack if one will do the job.
I'm sure there are extreme cases where Memcached is in fact the better tool for the job over Redis for caching workloads. I also expect that 99%+ of people trying to decide between Redis and Memcached will never get into that territory. Redis is so fast that unless you're doing O(n) operations on very large data sets, you're unlikely to notice any substantial differences.
The other thing about caching is that the data is, by its nature, disposable and rebuildable. So even in the extreme minority case where Redis would no longer be sufficient, migration from one KV cache system to another is about as easy as it gets. Pre-optimizing your caching layer stack for Facebook levels of traffic isn't even justifiable from a lock-in standpoint like it might be with other data storage needs.
In the case of your average Sidekiq user, serving cache fragments for a Rails app, memcached vs redis for your caching layer is almost certainly an inconsequential choice WRT the performance of your application, and the choice of Redis reduces your ops and client library overhead. The choice should be pretty clear in those circumstances.
Re: Clarifications about Redis and Memcached
#17Last time I looked Redis would balk if it ran out of memory whilst memcached just pushed out the oldest caches.
I guess that's not a problem anymore..?
Re: Clarifications about Redis and Memcached
#18Last time I looked Redis would balk if it ran out of memory whilst memcached just pushed out the oldest caches. I guess that's not a problem anymore..?
By changing the eviction policy and using the maxmemory directive, you can effectively reproduce memcached's behavior in redis.
Re: Clarifications about Redis and Memcached
#19If data access is equally distributed across keys, then running multiple Redis instances (with sharding) is perfect, since ultimately you want each key to be serviced by a single core, for optimal performance in a system with caches and NUMAs. However, if data access is read-heavy and not equally distributed across keys, then having shared memory (aka multithreading) is quite essential since you want all cores to ope…
In a networked server getting data from memory, the time required to access the data itself is negligible, so the real performance in mostly a factor of how much I/O a thread can sustain (Redis with heavy pipelining will handle at least 500k ops/sec in the same hardware it handles 100k ops/sec without pipelining). There is still the case of the load to be very biased towards, like, 5% of keys. But that 5% of keys are very very likely to be distributed across all processes.
However this is not true when you have a case with, like, 2/3 super hot keys that are requested a lot more than any other. But in this case what allows scalability is replication with many read-replicas.
Re: Clarifications about Redis and Memcached
#20If data access is equally distributed across keys, then running multiple Redis instances (with sharding) is perfect, since ultimately you want each key to be serviced by a single core, for optimal performance in a system with caches and NUMAs. However, if data access is read-heavy and not equally distributed across keys, then having shared memory (aka multithreading) is quite essential since you want all cores to ope…
Good point, but I could add that case 1 is vanishingly unlikely to happen in a real system. You always have hot keys, i.e. a Zipfian distribution. That's almost tautological for a cache -- by using a small amount of space you can handle most of the reads. So, not knowing much about Redis, I would conclude based on this blog post that memcached has a pretty big advantage as a cache in real systems (multithreading).
I think this is not the case, I already replied to the parent comment.