Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis Cluster Tutorial

redis.io

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Redis Cluster Tutorial

#21

Post by seiji »

taf2 wrote: post #20

I wish redis had synchronous write option to guarantee that when a write response is received that all nodes in the cluster have also received a copy. Even if this was just an ack that data is replicated into memory instead of disk. Sure there are cases where you'd rather operate with async, but really I would prefer to put haproxy in front of 3 - 6 redis nodes and let it load balance both reads and writes similar to…

That's in the plan, it's just not implemented yet. (It's implied under the Note: Redis Cluster in the future will allow users to perform synchronous writes when absolutely needed.)

Something like:

   SYNC 3
   SET key value
   [replies with success only after it's ACK'd on three cluster instances.]

Re: Redis Cluster Tutorial

#22

Post by bsg75 »

stiff wrote: post #4

If Redis would now just use names for databases instead of numbers and get rid of the limit of number of databases... Managing multiple application instances (production, test, staging for multiple countries) using a single shared Redis instance but requiring separate databases is a huge pain.

Along this line I would find named Lua scripts useful, so multiple clients could access script functionality without uploading themselves, or having to retrieve the SHA1 from a dictionary.

This idea may of course be contrary to the expected use of Lua scripts in Redis, and an attempt for the RDBMS part of my brain to look at Lua scripts as stored procedures.

Re: Redis Cluster Tutorial

#23

Post by bsg75 »

dberg wrote: post #13

seems like redis cluster is about turning redis into riak

In what way?

Some variant of clustering is a very common feature among many different database platforms.

Redis adding clustering does not make it a step towards turning it into Riak any more than it is a step towards turning it into MySQL [1]

[1] http://www.mysql.com/products/cluster/

Re: Redis Cluster Tutorial

#24

Post by taf2 »

seiji wrote: post #21
taf2 wrote: post #20

I wish redis had synchronous write option to guarantee that when a write response is received that all nodes in the cluster have also received a copy. Even if this was just an ack that data is replicated into memory instead of disk. Sure there are cases where you'd rather operate with async, but really I would prefer to put haproxy in front of 3 - 6 redis nodes and let it load balance both reads and writes similar to…

That's in the plan, it's just not implemented yet. (It's implied under the Note: Redis Cluster in the future will allow users to perform synchronous writes when absolutely needed. ) Something like: SYNC 3 SET key value [replies with success only after it's ACK'd on three cluster instances.]

I would definitely take slower write speed to have this extra assurance of high availability with data consistency... as it is today - it's pretty hard operate redis in a HA way even with sentinel... I'm sure others have figured it out but for me the best I have is a 30 second window when sentinel is switching the backup slave to master and either writes are lost haproxy can hold but it works sometimes... I can imagine with synchronous replication we'd have slower write speed but at least failover could be sub-second and possibly zero downtime... using a vip/keepalived - Still redis is such a good datastore, it's difficult to imagine a world/service without it. for what it's worth this looks to be the best setup i've found: http://failshell.io/sensu/high-availability-sensu/ would be interested if others have other setups that possibly work better...

Re: Redis Cluster Tutorial

#25

Post by antirez »

taf2 wrote: post #24
seiji wrote: post #21

Earlier quoted context omitted.

That's in the plan, it's just not implemented yet. (It's implied under the Note: Redis Cluster in the future will allow users to perform synchronous writes when absolutely needed. ) Something like: SYNC 3 SET key value [replies with success only after it's ACK'd on three cluster instances.]

I would definitely take slower write speed to have this extra assurance of high availability with data consistency... as it is today - it's pretty hard operate redis in a HA way even with sentinel... I'm sure others have figured it out but for me the best I have is a 30 second window when sentinel is switching the backup slave to master and either writes are lost haproxy can hold but it works sometimes... I can imagi…

Hey, going a bit off topic about Sentinel, this should not happen. Sentinel + Redis + Clients as a distributed system is not able to guarantee strong consistency with arbitrary partitions clearly, but in the scenario you describe, that is, the master crashes or is partitioned away alone, the window should be like a fraction of a millisecond (the replication link latency) most of the times.

EDIT: to clarify:

1) Sentinels will tell clients master is A.

2) A fails.

3) Sentinels will still tell it is A before failover.

4) Failover happens.

5) Sentinels will finally tell master is B.

Re: Redis Cluster Tutorial

#26

Post by taf2 »

antirez wrote: post #25
taf2 wrote: post #24

Earlier quoted context omitted.

I would definitely take slower write speed to have this extra assurance of high availability with data consistency... as it is today - it's pretty hard operate redis in a HA way even with sentinel... I'm sure others have figured it out but for me the best I have is a 30 second window when sentinel is switching the backup slave to master and either writes are lost haproxy can hold but it works sometimes... I can imagi…

Hey, going a bit off topic about Sentinel, this should not happen. Sentinel + Redis + Clients as a distributed system is not able to guarantee strong consistency with arbitrary partitions clearly, but in the scenario you describe, that is, the master crashes or is partitioned away alone, the window should be like a fraction of a millisecond (the replication link latency) most of the times. EDIT: to clarify: 1) Sentin…

could be something that haproxy is causing - when switching masters... maybe I need a script that handles switching the vip when sentinel detects failure in master instead of haproxy with a backup... btw - thank you for redis.

Re: Redis Cluster Tutorial

#27

Post by seiji »

taf2 wrote: post #26
antirez wrote: post #25

Earlier quoted context omitted.

Hey, going a bit off topic about Sentinel, this should not happen. Sentinel + Redis + Clients as a distributed system is not able to guarantee strong consistency with arbitrary partitions clearly, but in the scenario you describe, that is, the master crashes or is partitioned away alone, the window should be like a fraction of a millisecond (the replication link latency) most of the times. EDIT: to clarify: 1) Sentin…

could be something that haproxy is causing - when switching masters... maybe I need a script that handles switching the vip when sentinel detects failure in master instead of haproxy with a backup... btw - thank you for redis.

You shouldn't need haproxy in the loop at all.

If your redis client is Sentinel-aware, you ask your client directly "give me the redis master server" and it asks Sentinel which server is master. The only ip address involved for your configuration is the address of the sentinel(s).

haproxy will take 30 seconds to detect your failure. Sentiel will notify you immediately. Sentinel is the one determining the exact state of the master election, and it'll be very chatty when a new master replaces the old one.

Re: Redis Cluster Tutorial

#28

Post by mbell »

ransom1538 wrote: post #8

I have a few serious concerns: Suppose have nodes A,B,C,D running with slaves and you have a user u1 start retrieving and storing data. The odds will be high that the user u1 stores keys on all nodes A,B,C,D since all the keys are based on a numerical hashed slot. 1) The user u1 will end up connecting to all nodes. Thus as you scale adding an "E" you always end up connecting to each cluster having the same amount of…

In general I think you're right but I'm not sure why you would have 5,000 connections to each node. There isn't any reason I can think of to maintain more than 1 connection per client (application process) to each node. You don't gain anything by 'connection pooling' with Redis so it's not like each app server would want to keep a 100 connection pool around. I guess you could have an issue if you have 5000 application processes all wanting to connect to a single redis cluster, but I'd argue that you'd be well past the point of needing to break your system into services by then anyway.

Re: Redis Cluster Tutorial

#29

Post by heterogenic »

One thing I'd love to see (perhaps in a future iteration?) is a way to have overlapping ranges for redundancy, instead of mirrors.

The big benefit to this is when balancing write-heavy workloads, when the slave would not be getting its share of the total load.

I believe this could be accomplished by running a second DB on each machine as a slave to the next in the cycle, and that's what I'll probably try when we scale up to needing a cluster. But having it be a supported scenario (or facilitated somehow) would make me feel much better.

(edit: Just a feature request of course, exciting to see this release! Thanks for all your hard work antirez!)

Post reply on HN