Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to do distributed locking

martin.kleppmann.com

Post reply on HN
Print view View original

51–60 of 73 posts

Re: How to do distributed locking

#51

Post by wyaeld »

ploxiln wrote: post #42
antirez wrote: post #33

Earlier quoted context omitted.

Hello ploxiln, to see systems with clocks jumping is indeed possible, I'll make it more clear in my blog post, but basically it is possible with some effort to prevent this in general, but more specifically the Martin suggestion of using the monotonic time API is a good one and was already planned. It's very hard to argue that the monotonic time jumps back and forth AFAIK, so I think it's a credible system model. The…

It's fair to criticize my somewhat harsh and yet hard-analysis-free critique. But let me explain why it matters to me, and what the relation to Redis Sentinel is: "Redlock" is a fancy name, involves multiple nodes and some significant logic, less experienced developers will assume it's bulletproof. "Redis Sentinel" is a fancy name, involves multiple nodes and some significant logic, less experienced developers will a…

I have to disagree here.

"fancy names == people think bulletproof" is not a credible criticism.

Now "Oracle Enterprise Manager Fusion Middleware Control" is a fancy name!!! Pretty sure no-one with a clue thinks a product named like that is bulletproof, and its probably got significant logic/nodes etc

I apparently have the opposite experience to you. I'll often find complex distributed systems that are painful to troubleshoot when they misbehave, and find it refreshing on the other hand when someone is just using Redis, because typically THAT system is working a lot more predictable.

If people are too inexperienced to realize that ALL systems have tradeoffs, and not read up on what those are (because apparenly the name is fancy) then they'll get burned. Antirez does a pretty good job of explaining and documenting where he sees Redis limits to be.

Arguing that everyone should be home-baking their own failovers until of learning the limits of well-known ones our there doesn't seem like responsible advice.

Re: How to do distributed locking

#52

Post by batbomb »

kang wrote: post #45

> If they could, distributed algorithms would do without clocks entirely, but then consensus becomes impossible Consensus can be achieved without clocks using blockchains: Instead of timelocking the resource, client can broadcast his altered version after making changes to the network. The next client than then start working on top of this changed resource, but since multiple versions might be floating around (due to…

The blockchain is a consensus protocol.

Re: How to do distributed locking

#53

Post by Randgalt »

mattiemass wrote: post #2

Interesting stuff. In all my distributed systems work so far, I've assumed that a distributed lock is a thing to avoid. I really should take another look at them, just as a tool to have at my disposal.

If you assume that your distributed lock gives you transactional guarantees that you are the only lock holder then you are making a mistake. If, however, you can tolerate small overlaps in lock holders you are fine and this helps with numerous distributed algorithms. Further, using other facilities such as fences can make it even more secure. Another feature of ZooKeeper is write-with-version. You could obtain a lock (using Apache Curator - note I wrote this), then do a write-with-version to achieve 100% certainty that you are the only writer.

Re: How to do distributed locking

#54

Post by kang »

querulous wrote: post #49
kang wrote: post #45

> If they could, distributed algorithms would do without clocks entirely, but then consensus becomes impossible Consensus can be achieved without clocks using blockchains: Instead of timelocking the resource, client can broadcast his altered version after making changes to the network. The next client than then start working on top of this changed resource, but since multiple versions might be floating around (due to…

a blockchain does not make consensus without clocks possible it is just a grossly inefficient vector clock implementation (that incidentally does not provide consensus in the distributed systems sense)

I just described how you can do it without clocks. Is there a mistake somewhere, sorry?

Re: How to do distributed locking

#55

Post by querulous »

kang wrote: post #54
querulous wrote: post #49

Earlier quoted context omitted.

a blockchain does not make consensus without clocks possible it is just a grossly inefficient vector clock implementation (that incidentally does not provide consensus in the distributed systems sense)

I just described how you can do it without clocks. Is there a mistake somewhere, sorry?

a clock in the context of a distributed system is a method of providing a partial ordering (a happened before b). most blockchain implementations use a merkle tree that uses predecessor transactions as inputs to the output as the clock

regardless, blockchains as implemented in certificate transparency and bitcoin aren't actually consensus systems, they are just (highly accurate) approximations

Re: How to do distributed locking

#56

Post by kang »

querulous wrote: post #55
kang wrote: post #54

Earlier quoted context omitted.

I just described how you can do it without clocks. Is there a mistake somewhere, sorry?

a clock in the context of a distributed system is a method of providing a partial ordering (a happened before b). most blockchain implementations use a merkle tree that uses predecessor transactions as inputs to the output as the clock regardless, blockchains as implemented in certificate transparency and bitcoin aren't actually consensus systems, they are just (highly accurate) approximations

Both of the points you make are true but how are they relevant here?

We are talking about clocks that tell time in the quoted quote (time is ordering of events by definiton) & consensus in bitcoin is final(accurate) enough an practical approximation as much as a centralised system as described in OP is. Either I am misunderstanding or this is your dislike for bitcoin.

Re: How to do distributed locking

#57

Post by sdab »

anon987 wrote: post #50
sdab wrote: post #28

"When used as a failure detector, timeouts are just a guess that something is wrong. (If they could, distributed algorithms would do without clocks entirely, but then consensus becomes impossible [10]." Having just re-read Lynch's paper, can you explain what you mean here? I didn't see anything explicitly relying on time. It could be there is some implicit usage I didnt see. Additionally, the paper's impossibility re…

"When used as a failure detector, timeouts are just a guess that something is wrong." I send a packet over the network. I configured the timeout to be 10 seconds. 10 seconds passes. Is the packet truly lost? Or, maybe the packet was received, processed, and and a response is on the way back, but it takes 20 seconds instead of 10! Or maybe it takes an hour instead of 20 seconds. Or a decade. We don't know if the packe…

My question was about his claim in regards to the result in the paper. In practice, you are right but a consensus algorithm is still "partially correct" if it never decides on a value. It is only incorrect if different nodes decide on different values. For example, paxos does is not guaranteed to decide on a value. So timeouts are useful guessing tools, but I dont see how there is no consensus possible without them.

Re: How to do distributed locking

#58

Post by EGreg »

Why would you ever want to do distributed locking when you can do MVCC via eg vector clocks, possibly with cryptographic signatures?

If you want to acquire some shared resource, then elect an owner for that resource.

Re: How to do distributed locking

#60

Post by jondubois »

TallGuyShort wrote: post #34
cryptica wrote: post #8

If you want to have a truly parallel system, you cannot share resources between processes. If you need a lock, then that implies sharing of resources... Which implies limited scalability according to Amdahl's law. To achieve unlimited scalability, you need to make sure that no single data store is shared between all processes and that the amount of interprocess communication (on a per-process basis) doesn't increase…

Although no one starts out wanting to have a truly parallel system. They start out wanting to solve a problem and when that requires scalability or redundancy a parallel system is a means to that end. Not every problem can or should be solved with no shared state, and when that's the case, you should know how to do it right.

I find the code of highly parallel systems much more elegant and easy to read.

Personally, I find it easier to write highly parallel code than parallel-up-to-a-certain-point code that involves locks likes mutexes and/or semaphores. Highly parallel code just takes a little bit of extra planning and thinking up-front but it saves lots of time and stress down the line.

Post reply on HN