Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis cluster, no longer vaporware

antirez.com

Post reply on HN
Print view View original

31–40 of 52 posts

Re: Redis cluster, no longer vaporware

#31

Post by jeremyjh »

seiji wrote: post #16
swartkrans wrote: post #14

Earlier quoted context omitted.

Seems like given the above problems you would only want to use this with non-critical data of which you didn't care a great deal about its integrity. This doesn't describe a lot of use cases. It's not too hard to get memcached running for cluster of machines, nor is it really all that difficult to get started with cassandra for basic applications, although I haven't done it. I've heard ReThinkDB wants to be a serious…

This doesn't describe a lot of use cases. Do you have any idea how many people run master/slave mysql? That's async and will definitely give you data drift over time. There's an entire suite of Percona tools to detect and repair broken mysql datasets because mysql breaks data so often due to replication problems. The world is much more async and much less consistent than anybody realizes (cash machines are even async…

Cash machines are definitely not async in normal operations, though the authorization network is configured with rules for stand-in mode when third-party systems are unavailable. Without the primary authorization system immediately available the ATM is hard down.

Re: Redis cluster, no longer vaporware

#32

Post by antirez »

peterwwillis wrote: post #29

@antirez, what would you say is the most useful lesson you've learned throughout this process?

Argument with people just with technical arguments, don't get into Twitter fights, and at the end of the day, use the feedbacks as a learning process, but do what you believe makes sense. So a social lesson, more than technological.

Re: Redis cluster, no longer vaporware

#33

Post by bkirwi »

antirez wrote: post #22
bkirwi wrote: post #20

Earlier quoted context omitted.

In theory, Jepsen or a Jepsen-like system should be able to check any of these failure modes. On the other hand, it sounds like Redis Cluster offers few hard guarantees; instead, it promises that failures should be rare 'in practice'. Which is a fine thing for a tool to do, of course, but it makes things less amenable to the kind of stress-testing Jepsen does -- since running inside Jepsen's little universe is about…

Oh, there are definitely hard rules like that. For example a majority partition never accepts queries, and when there are no partitions at all Redis Cluster guarantees to converge on a single value for each key, and to a single view of the cluster configuration. I'll try to document better this things, but basically they arise from the simple algorithm that makes the configuration eventually consistent.

Neat; that should be very useful.

It's great to see that Redis has a official story for clustering / failover out; like you said in the post, the worst distributed systems are the ones you have to rewrite every single time. It's going to be interesting watching this evolve.

Re: Redis cluster, no longer vaporware

#35

Post by mfenniak »

seiji wrote: post #11
mfenniak wrote: post #6

Earlier quoted context omitted.

I have very little evidence to support this, but they probably use twemproxy. twemproxy is a proxy for redis that automatically manages routing commands to multiple redis servers efficiently. They definitely don't use redis cluster, because redis cluster requires explicit client-side support. That support is only available in a limited number of clients today.

Nope, they don't (and neither should anybody else... twemproxy is weird).

Oh, weird in what way? I was considering using it for horizontal redis scaling recently, under the impression that redis cluster was further away from release than it seems to be.

Re: Redis cluster, no longer vaporware

#36

Post by seiji »

mfenniak wrote: post #35
seiji wrote: post #11

Earlier quoted context omitted.

Nope, they don't (and neither should anybody else... twemproxy is weird).

Oh, weird in what way? I was considering using it for horizontal redis scaling recently, under the impression that redis cluster was further away from release than it seems to be.

The code is just strange. To support Redis, it has a 2,500 line Redis adapter. It has special cases for each Redis command because it doesn't extract key position information from Redis directly.

Plus, despite the name and where it's hosted, Twitter never used it in production for Redis. Redis support was just added "because we can."

It probably works for very direct use cases, but it's one of those things that could probably bite you very severely under edge cases, and you'd be left on your own to figure out what happened.

Re: Redis cluster, no longer vaporware

#37

Post by litewulf »

seiji wrote: post #36
mfenniak wrote: post #35

Earlier quoted context omitted.

Oh, weird in what way? I was considering using it for horizontal redis scaling recently, under the impression that redis cluster was further away from release than it seems to be.

The code is just strange. To support Redis, it has a 2,500 line Redis adapter. It has special cases for each Redis command because it doesn't extract key position information from Redis directly. Plus, despite the name and where it's hosted, Twitter never used it in production for Redis. Redis support was just added "because we can." It probably works for very direct use cases, but it's one of those things that could…

We run twemproxy over about a terabyte of data in Redis, and haven't had any problems...

Re: Redis cluster, no longer vaporware

#38

Post by IgorPartola »

antirez wrote: post #30
IgorPartola wrote: post #26

Earlier quoted context omitted.

From what I understand, there are two types of systems really (in terms of CAP): those that do async replication and those that do sync replication. Async replicated systems can be (but are not necessarily) AP: each system can process requests independently of the other, and they'll get consistent eventually (how they arrive at this consistency is different between systems, and is not always "correct" for the general…

Hello Igor, basically we can rule out synchronous systems: product mismatch from the POV of Redis. We are left with AP systems. AP systems trying to get as good as they can from the POV of Availability, and the consistency model provided (even if not "C"), do these two additional things that Redis Cluster is not able to do: 1) They are available in the minority partition. Redis Cluster has some degree of availability…

Thanks for the explanation. Is it possible to add something like "this record was not merged cleanly" flag? Basically, from what I understand without sync replication you cannot theoretically have a system that does the correct merge in all use cases. MySQL would deal with this by letting two replicas contain different data until you scan for this in your application code and detect it. Other systems (I think Cassandra) will attempt to merge things in the background after the partition is gone, but with strategies like "last write wins" or even versioning you can still get bad records. I am thinking of a system that can do whatever strategy you choose, or even just "last write wins" but warns you that a write on this key was lost. That way you can either ignore it if that's appropriate (some types of sensors for example, that frequently update values) or you can bubble this error up your application code until a user can fix it. On the other hand I definitely can see this as a performance issue so it might be more appropriate for a datastore that is aiming for high consistency while still allowing AP mode.

Re: Redis cluster, no longer vaporware

#39

Post by zamio »

seiji wrote: post #36
mfenniak wrote: post #35

Earlier quoted context omitted.

Oh, weird in what way? I was considering using it for horizontal redis scaling recently, under the impression that redis cluster was further away from release than it seems to be.

The code is just strange. To support Redis, it has a 2,500 line Redis adapter. It has special cases for each Redis command because it doesn't extract key position information from Redis directly. Plus, despite the name and where it's hosted, Twitter never used it in production for Redis. Redis support was just added "because we can." It probably works for very direct use cases, but it's one of those things that could…

pinterest, snapshot, vine, wanelo use the twemproxy in prod for redis

Re: Redis cluster, no longer vaporware

#40

Post by seiji »

zamio wrote: post #39
seiji wrote: post #36

Earlier quoted context omitted.

The code is just strange. To support Redis, it has a 2,500 line Redis adapter. It has special cases for each Redis command because it doesn't extract key position information from Redis directly. Plus, despite the name and where it's hosted, Twitter never used it in production for Redis. Redis support was just added "because we can." It probably works for very direct use cases, but it's one of those things that could…

pinterest, snapshot, vine, wanelo use the twemproxy in prod for redis

Well, companies also use ejabberd, but that doesn't make ejabberd a valid piece of software either.

Then there's the massive problem of "everyone" using MongoDB and we know how much pain _that_ sucker causes. Playing popularity contests with software isn't useful. People install software either because of social reasons ("I don't know how anything works, but I hear everybody is using X, so I'm going to use X too!") or because they need the functionality ("redis is AMAZING at 30 different things!").

If install base at well known companies were a valid measure, we'd all be using Windows laptops while talking into our Kin and squirting our Zunes all over each other.

Post reply on HN