Live data from Hacker News

The Redis criticism thread

antirez.com

31–40 of 47 posts

Re: The Redis criticism thread

#31
post #25
post #23

Earlier quoted context omitted.

I think Antirez is simply doing a master/slave system with async replication. This is what mysql, solr, elasticsearch, kafka, etc do. This is not an unusual model at all. In CAP theorem terms, Redis has picked zero (remember CAP theorem is pick at most two). There's a bunch of people who've made this choice, but why? C incurs a synchronization cost. A means that you have to reconcile different writestreams. If you wa…

I assert that this design actually allows for arbitrarily long windows of data loss, but I haven't verified that the implementation matches my understanding of antirez's WAIT/failover algorithm yet. Pretty sure though.

If you read the Redis Cluster documentation, the data loss is acknowledged, even explaining what are the most obvious failure modes where this happens. WAIT does not provide strong consistency either, and is actually not even documented in the Cluster doc. However WAIT as it is makes the possibility for data loss less likely.

Once everything in the Redis Cluster design and documentation is about trading consistency for performance, I would expect the system to be analyzed for what it is: is it good at providing weak but practically usable consistency and performances? In short, it does respect the design intent?

Saying again and again that it features not consistency is a sterile exercise.

Re: The Redis criticism thread

#32
post #23
post #16

To summarize quickly for those that didn't read the article, this focuses mostly on the new "clustering" aspect not on the "classic" single Redis server (if you wish). I think it is important to be honest with the users and make it clear how and what happens behind the scenes, how data could be lost. And Salvatore has done most of this, maybe just make it a bit more explicit, as there still seems to be some confusion…

I think Antirez is simply doing a master/slave system with async replication. This is what mysql, solr, elasticsearch, kafka, etc do. This is not an unusual model at all. In CAP theorem terms, Redis has picked zero (remember CAP theorem is pick at most two). There's a bunch of people who've made this choice, but why? C incurs a synchronization cost. A means that you have to reconcile different writestreams. If you wa…

In CAP theorem terms, Redis has picked zero (remember CAP theorem is pick at most two).

That might be true, but describing something in the CAP framework is not the only salient thing you can say about a distributed storage system. It is characterizing the failure modes.

You also have to think about what happens in the normal case. In the normal case you have a consistency vs. latency tradeoff. People have written about this, but unfortunately I don't think this broader way of thinking hasn't the attention it deserves:

http://dbmsmusings.blogspot.com/2011/12/replication-and-late...

http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-an...

"If you examine PNUTS through the lens of CAP, it would seem that the designers have no idea what they are doing (I assure you this is not the case). Rather than giving up just one of consistency or availability, the system gives up both!"

"The reason is that CAP is missing a very important letter: L. PNUTS gives up consistency not for the goal of improving availability. Instead, it is to lower latency. "

Re: The Redis criticism thread

#33
post #22

Earlier quoted context omitted.

I don't know, I don't think it should be compared (or considered as an alternative to) MongoDB, and certainly never SQL. I use it to store cached keys, session data and similar other data that I can always regenerate. I've also had great success using it as a message queue on multiple projects. What was your use case, and what was your experience with it?

Agreed. If Redis is an acceptable alternative to MongoDB you're using one of the two very incorrectly.

Use case: I want to store some json values against some string keys. Which would be very incorrect to use for this?

Re: The Redis criticism thread

#35
post #7

Earlier quoted context omitted.

I don't pay super close attention to this stuff, but it's very obvious to me that not everyone likes Redis; in particular, the kinds of people who have problems with Mongodb tend also to have the same problems with Redis; long-term operational reliability is a big question mark with it. For my part: on only a few occasions (mostly fuzzer farm stuff) have I ever used Redis and been happy with the decision. I usually r…

I don't know, I don't think it should be compared (or considered as an alternative to) MongoDB, and certainly never SQL. I use it to store cached keys, session data and similar other data that I can always regenerate. I've also had great success using it as a message queue on multiple projects. What was your use case, and what was your experience with it?

Redis people think it could be used instead of SQL for some designs:

http://redis.io/topics/twitter-clone

I want to use it in a small project so I can learn the design patterns associated with key-value stores. I'm sure there are good and bad things about using Redis vs Postgres.

But it has to hold my data, so... does it hold my data or not? I'm talking about single box stuff here, of course.

(Of course, I'll just use it and find out myself).

Re: The Redis criticism thread

#36
post #22

Earlier quoted context omitted.

I don't know, I don't think it should be compared (or considered as an alternative to) MongoDB, and certainly never SQL. I use it to store cached keys, session data and similar other data that I can always regenerate. I've also had great success using it as a message queue on multiple projects. What was your use case, and what was your experience with it?

Agreed. If Redis is an acceptable alternative to MongoDB you're using one of the two very incorrectly.

MongoDB is not significantly better at any particular thing so yes, Redis can be used as an alternative for the things I may use MongoDB for. Keep a close eye on http://www.amisalabs.com

Re: The Redis criticism thread

#37
post #35

Earlier quoted context omitted.

I don't know, I don't think it should be compared (or considered as an alternative to) MongoDB, and certainly never SQL. I use it to store cached keys, session data and similar other data that I can always regenerate. I've also had great success using it as a message queue on multiple projects. What was your use case, and what was your experience with it?

Redis people think it could be used instead of SQL for some designs: http://redis.io/topics/twitter-clone I want to use it in a small project so I can learn the design patterns associated with key-value stores. I'm sure there are good and bad things about using Redis vs Postgres. But it has to hold my data, so... does it hold my data or not? I'm talking about single box stuff here, of course. (Of course, I'll just us…

That's supposed to showcase redis' features, not to tell you that you should dump Postgres for redis.

Re: The Redis criticism thread

#38
post #33
post #22

Earlier quoted context omitted.

Agreed. If Redis is an acceptable alternative to MongoDB you're using one of the two very incorrectly.

Use case: I want to store some json values against some string keys. Which would be very incorrect to use for this?

If the data is critical and you must keep them, both. If not, either.

Re: The Redis criticism thread

#39
Quick things

1) I expected that thread to look like a Cultural Revolution struggle session". Thankfully it wasn't.

2) As I am sure many others have already said it, durability has very little to do with CAP. CAP is about the A and I in in ACID, D is an orthogonal concern.

3) Durability doesn't necessarily mean losing high performance. Most databases let the user choose how much data they're willing to lose and for what latency decreases -- the standard approach (used even in main-memory databases like RAMCloud[1] and recent versions of VoltDB[2]) is to keep a separate write-ahead log (WAL) and let the end-user choose how frequently to fsync() it to disk as well as how frequently to flush a snapshot of main memory data structures to disk.

There are many papers (e.g., http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.174....) that talk about various challenges of building WALs, but fundamentally users who want strongest possible single-machine durability can choose to fsync() on every write (and usually use battery backed raid controllers or separate log devices like SSDs with supercapacitors or even NVRAM if the writes are going to be larger than what fits into the RAID controller's write-back cache). Others can choose to live with possibility of losing some writes, but use replication[3] to protect against power supply failure and crashes -- idea being that machines in a datacenter are connected to a UPS, replicas don't all live on the same rack (to protect against -- usually rack local -- UPS failure), and there's cross-data center replication (usually asynchronous with possibility of some conflicts -- notable exception being Google's Spanner/F1) to protect (amongst many other things...) against someone leaning against the big red-button labeled "Emergency Power Off" (which is exactly what you think it is).

Flushes of main data do also hurt bandwidth with spinning disks and old or cheap SSDs, but there's a solution: use a a good, but commodity MLC SSD with synchronous toggle NAND flash with a good controller/firmware (Sandforce SF2000 or later series, Intel/Samsung/Indilinx's recent controllers) -- these work on the same principle as DDR memory (latch onto both edges of a signal) to provide sufficient bandwidth to handle both random reads (traffic you're serving) and sequential writes (the flush).

4) I known several tech companies and/or engineering departments therein who absolutely love and swear by redis. There are very good reasons for it: the code is extremely clean and simple[4] and it handles a use case that neither conventional databases nor pure kv-stores or caches handle well.

That use case is roughly described data structures on an outsourced heap for maintaining a materialized view (such as a user's newsfeed, adjacency lists of graphs stored efficiently using compressed bitmaps, counts, etc...) on top of a database. So my advise to antirez is to focus the effort around making this use case simpler rather than build redis out into a database: build primitives to let developers piggy back durability and replication to a database or a message queue. In fact, I've known of multiple startups that have (in an ad-hoc way) implement pretty much exactly that.

This is still a tough problem, but one which (I think) would yield a lot more value to redis users. Just thinking out loud, one approach could be a way to associate each write to redis with an external transaction id (HBase LSN, MySQL gtid, or perhaps an offset in a message queue like Kafka). When redis flushes its data structures to disk, it stores the last flushed transaction id to persistent storage.

I would also implement fencing within redis: when in a "fenced" mode redis won't accept any requests on a normal port, but can accept writes through an bulk batch update interface that users can program against. This could be more fine grained by having both a read-fence and a write fence, etc...

This makes it easier for users to tackle replication and durability themselves:

For recovery/durability, users can configure redis such that after a crash, it is automatically fence and "calls-back" with that last flushed id into users' own code -- by either invoking a plugin, doing an REST or RPC call to a specified endpoint, or simply using fork() and executing a user configured script which would use the bulk API.

For replication, users could use a high-performance durable message queue (something I'd imagine some users already do) -- a (write-fenced) standby redis node can then become a "leader" (unfence itself) once its caught up to the latest "transaction id" (last consumed offset in the message queue, as maintained by the message queue itself -- in case of Kafka this is stored in ZooKeeper). More advanced users can tie this with database replication by either tailing the database's WAL (with a way to transform WAL edits into requests to redis) or using a plugin storage engine for the database.

Fundamentally, where I see redis used successfully are uses cases where (prior to redis) users would use custom C/C++ code. This cycles back to the "outsourced on heap data structures" idea -- redis lets you use a high level languages to do fast data manipulation without worrying about performance of the code (especially if using a language like Ruby or Python) or garbage collection on large heaps (a problem with even the most advanced concurrent GCs like Java's).

There have been previous attempts to build these outsourced heaps as end-to-end distributed system that handle persistence, replication, and scale-out and transactions. These are generally called "in-memory data grids" -- some simply provide custom implementations of common data structures, others act almost completely transparent and require no modifications to the code (e.g., some by using jvmti). Terracotta is a well known one with a fairly good reputation (friends who contract for financial institutions and live in hell^H^H^H^H world of app servers and WAR files swear by it), but JINI and JavaSpaces were some of the first (too came too early, way before the market was ready) and are rightly still covered by most distributed systems textbooks. However their successful use usually requires Infiniband or 10GbE (or Myrinet back in dotcom days) -- reliable low-latency message delivery is needed as (with no API to speak off) there's no easy way for users to recover from network failures or handle non-atomic operations.

To sum it up, I'd suggest to examine and focus on use-case where redis is already loved by its users, don't try to build a magical end to system as it won't conserve the former, and make it easy (and to an extent redis already does this) to let users build custom distributed systems with redis as a well-behaved component (again, they're already doing this).

[1] https://ramcloud.stanford.edu/wiki/display/ramcloud/Recovery

[2] http://voltdb.com/intro-to-voltdb-command-logging/

[3] Whether it's synchronous or not is about the atomicity guarantees and not durability -- the failure mode of acknowledging a write and then 'forgetting' can happen in these systems even if they fsync every write.

[4] It reminds me of NetBSD source code: I can open up a method and it's very obvious what it does and how.

Re: The Redis criticism thread

#40
post #27

Just to be clear, the criticism is of Redis Cluster, which has been redesigned because the first design was so lax it was effectively unusable. The real issue, is that as of right now I have no idea what the exact semantics and failure conditions of Redis cluster are. I'm not clear that anyone really knows. Salvatore thinks he knows, but we can't be sure that he does. With all this talk of practicality, what really m…

He is using known algorithms. He describes them here: https://news.ycombinator.com/item?id=6780342 .

see Aphyr's comments here: https://groups.google.com/d/msg/redis-db/Oazt2k7Lzz4/q9YDD7D...
Post reply on HN