How to do distributed locking (2016) • Hacker News Board

Re: How to do distributed locking (2016)

#11

Post by diggs » Sat, May 13, 2023, 12:49 AM UTC

klysm wrote: post #7

FooBarWidget wrote: post #2

I also wrote about distributed locking a while ago, particularly about implementing one on Google Cloud Storage. https://www.joyfulbikeshedding.com/blog/2021-05-19-robust-di... This is useful because cloud storage is very cheap and serverless. It certainly beats running a Redis or PostgreSQL instance. In my research and implementation I took care of the problems described in this article (as well as the problems I en…

It’s just not possible to implement a correct system using distributed locks with these types of semantics.

Every time this topic comes up, it seems people invariably divide into one of two groups. In the first group are those who trust in the odds and play the probability game. In the other, those who demand absolute guarantees. Neither group seems fully capable of understanding the other's standpoint.

Let me add my own perspective: Distributed locks are a fallacy. They can be beneficial in decreasing contention, under the assumption that "most of the time, only one actor will be active". However, by themselves, they offer no solid guarantees. The blog post addresses this point by introducing the concept of fencing tokens. These tokens have the potential to provide concrete guarantees, but they require the cooperation of downstream systems for enforcement, which isn't always possible.

I was really surprised to see Antirez argue for the probability approach.

Re: How to do distributed locking (2016)

#12

Post by rubiquity » Sat, May 13, 2023, 1:20 AM UTC

antirez wrote: post #6

Please note that I replied to this blog post, when it was published. You may be interested in reading the counter arguments. The author of the original post acknowledged that at least one of his claim was wrong, but I don't know if they modified the post later. Anyway we gained some time perspective, too: Redlock was massively used in the last 10 years, and I have never heard of its properties failing in the practice…

> Redlock was massively used in the last 10 years, and I have never heard of its properties failing in the practice: that is, no incidents so far.

I enjoyed these posts between yourself and Martin. Don’t be too confident though. Data races almost always go unnoticed.

Re: How to do distributed locking (2016)

#13

Post by FooBarWidget » Sat, May 13, 2023, 6:42 AM UTC

diggs wrote: post #11

klysm wrote: post #7

Earlier quoted context omitted.

It’s just not possible to implement a correct system using distributed locks with these types of semantics.

Every time this topic comes up, it seems people invariably divide into one of two groups. In the first group are those who trust in the odds and play the probability game. In the other, those who demand absolute guarantees. Neither group seems fully capable of understanding the other's standpoint. Let me add my own perspective: Distributed locks are a fallacy. They can be beneficial in decreasing contention, under th…

I don't know why you say neither group understands each other when I literally said in my comment that my approach is probabillistic.

Also, how is configuring an infinite timeout (and checking whether the client is actually gone before manually breaking the lock lease) not an absolute guarantee?

Re: How to do distributed locking (2016)

#14

Post by antirez » Sat, May 13, 2023, 7:22 AM UTC

rubiquity wrote: post #12

antirez wrote: post #6

Please note that I replied to this blog post, when it was published. You may be interested in reading the counter arguments. The author of the original post acknowledged that at least one of his claim was wrong, but I don't know if they modified the post later. Anyway we gained some time perspective, too: Redlock was massively used in the last 10 years, and I have never heard of its properties failing in the practice…

> Redlock was massively used in the last 10 years, and I have never heard of its properties failing in the practice: that is, no incidents so far. I enjoyed these posts between yourself and Martin. Don’t be too confident though. Data races almost always go unnoticed.

Thank you. I agree about races going unnoticed. And indeed my post tryed to address why the failure modes outlined in the original post are not possible (but for the monothonic timer issue, but this was fixed in Redis at some point).

Re: How to do distributed locking (2016)

#15

Post by klysm » Sat, May 13, 2023, 8:47 PM UTC

FooBarWidget wrote: post #13

diggs wrote: post #11

Earlier quoted context omitted.

Every time this topic comes up, it seems people invariably divide into one of two groups. In the first group are those who trust in the odds and play the probability game. In the other, those who demand absolute guarantees. Neither group seems fully capable of understanding the other's standpoint. Let me add my own perspective: Distributed locks are a fallacy. They can be beneficial in decreasing contention, under th…

I don't know why you say neither group understands each other when I literally said in my comment that my approach is probabillistic. Also, how is configuring an infinite timeout (and checking whether the client is actually gone before manually breaking the lock lease) not an absolute guarantee?

I guess it’s an absolute guarantee, but you lose liveness unless there is some other party handling those cases where the client is truly gone

Re: How to do distributed locking (2016)

#16

Post by diggs » Sun, May 14, 2023, 7:15 AM UTC

FooBarWidget wrote: post #13

diggs wrote: post #11

Earlier quoted context omitted.

Every time this topic comes up, it seems people invariably divide into one of two groups. In the first group are those who trust in the odds and play the probability game. In the other, those who demand absolute guarantees. Neither group seems fully capable of understanding the other's standpoint. Let me add my own perspective: Distributed locks are a fallacy. They can be beneficial in decreasing contention, under th…

I don't know why you say neither group understands each other when I literally said in my comment that my approach is probabillistic. Also, how is configuring an infinite timeout (and checking whether the client is actually gone before manually breaking the lock lease) not an absolute guarantee?

An infinite timeout works in theory, but it is impractical at scale.

Re: How to do distributed locking (2016)

#17

Post by antirez » Sun, May 14, 2023, 2:27 PM UTC

diggs wrote: post #11

klysm wrote: post #7

Earlier quoted context omitted.

It’s just not possible to implement a correct system using distributed locks with these types of semantics.

Every time this topic comes up, it seems people invariably divide into one of two groups. In the first group are those who trust in the odds and play the probability game. In the other, those who demand absolute guarantees. Neither group seems fully capable of understanding the other's standpoint. Let me add my own perspective: Distributed locks are a fallacy. They can be beneficial in decreasing contention, under th…

Redlocks don't use anything proabilistic, not in the sense that the lock may fail to guarantee what it promises at random.

Re: How to do distributed locking (2016)

#18

Post by FooBarWidget » Mon, May 15, 2023, 7:05 PM UTC

diggs wrote: post #16

FooBarWidget wrote: post #13

Earlier quoted context omitted.

I don't know why you say neither group understands each other when I literally said in my comment that my approach is probabillistic. Also, how is configuring an infinite timeout (and checking whether the client is actually gone before manually breaking the lock lease) not an absolute guarantee?

An infinite timeout works in theory, but it is impractical at scale.

Okay but then I really don't know what point you're trying to make. Either an infinite timeout, or non-absolute mutual exclusion guarantee: we have to pick either one. The right choice depends on the workload. There is no perfect solution that lets you have your cake and eat it too.

Re: How to do distributed locking (2016)

#19

Post by klysm » Sun, May 21, 2023, 7:30 AM UTC

antirez wrote: post #17

diggs wrote: post #11

Earlier quoted context omitted.

Every time this topic comes up, it seems people invariably divide into one of two groups. In the first group are those who trust in the odds and play the probability game. In the other, those who demand absolute guarantees. Neither group seems fully capable of understanding the other's standpoint. Let me add my own perspective: Distributed locks are a fallacy. They can be beneficial in decreasing contention, under th…

Redlocks don't use anything proabilistic, not in the sense that the lock may fail to guarantee what it promises at random.

While that may be true, it’s generally not possible to use red locks to build a correct system is mutual exclusion is a strict requirement