Live data from Hacker News

What could Go wrong with a mutex

evilmartians.com

31–40 of 59 posts

Re: What could Go wrong with a mutex

#31
post #15
post #10

I would zoom in on the actual fix: shrinking the size of the critical section. When you use defer to release the lock, the entirety of your function becomes the critical section. If you add another bit of code that needs the same lock, your program goes boom. I've hit this same issue a few times when I write a new, naive HTTP handler function that needs concurrent map access, then throw apachebench at it before I com…

I think deferring unlocks should be more openly described as an anti-pattern. I do appreciate the convenience of defer generally speaking but it has two problems that specifically hinder effective use of mutexes: 1. it allows you to roll up additional code that doesn't need to be inside the mutex, thus keeping other threads waiting longer for an unlock 2. Benchmarks have shown[1][2] that using defer is actually more…

There are several patterns possible which resolve this problem, your third example actually shows that, as it uses a function which only does the container access in a separate function which uses defer - this guarantees that no other code is executed while the lock is held. Alternatively, you could use an anonymous function inside your function which immediately gets called and performs the locking, lookup and deferred unlocking.

As a Lisper, this situation of course screams for having a with-lock macro which allows to add an elegant, but safe primitive which mostly takes care of this. This would usually be implemented in terms of a function which performs the locking/unlocking and calls a function which executes the body. Fortunately, Go allows doing exactly that, so this pattern could be used, here an untested sketch:

  func withLock (l Lockable, f func()) {
     l.Lock()
     defer l.Unlock()
     f()
   }

  ...
  withLock(c, func() { c.Counters[]++})
  ...
Of course, this doesn't protect you doing some potentially blocking actions in the body of the function executed, but the leaner the syntax is and as the defer is guaranteed to be executed at the end of the body, the risk is greatly reduced.

Re: What could Go wrong with a mutex

#32
post #23

Earlier quoted context omitted.

> Only if you’re also concurrency free. If you can still have two actors waiting on one another, you’re letter-of-the-law deadlock-free, but your program is still in effect deadlocked. I'm not sure we're using the same definitions. If you're non-blocking you have system wide progress, I'm not sure your examples of two actors mutually waiting on each other qualify. In any case I think Pony qualifies as wait-free too,…

> I'm not sure your examples of two actors mutually waiting on each other qualify Of course it qualifies, that's a classical text book example of deadlock.

Isn't that a live lock?

Can anything really shield you from that? It seems nothing can, in the same way nothing can really shield you from an infinite loop.

Re: What could Go wrong with a mutex

#33
post #9

Earlier quoted context omitted.

Avoiding deadlocks is easy if you're lockfree (or otherwise non-blocking), the interesting bit is being data-race free: https://www.ponylang.io/discover/#what-makes-pony-different

> Avoiding deadlocks is easy if you're lockfree (or otherwise non-blocking) Only if you’re also concurrency free. If you can still have two actors waiting on one another, you’re letter-of-the-law deadlock-free, but your program is still in effect deadlocked. > the interesting bit is being data-race free: Data race freedom is much weaker (and easier) than deadlock freedom, or race condition freedom. Safe rust is free…

> If you can still have two actors waiting on one another

There's no way to actively wait for another actor (except for using callbacks). The only way for your situation to occur would be if one of the actors receives a message with an included callback, but decides to never call it.

Re: What could Go wrong with a mutex

#34
post #8
post #4

People talk about the lack of generics et al as being Go's downfall but personally what I've found to be the biggest annoyance is the very thing Go boasts to be good at: writing bug/race free multi-threaded code. Channels are vastly oversold given their actual suitability and worse still, add their own classes of race conditions (deadlocks) so you often fallback to using mutexes for those, hopefully rare, occasions y…

Is there a language where deadlocks are not possible ?

Unlikely in a general purpose language

I expect you have to go out of your way to make it happen in erlang and actor based systems

Re: What could Go wrong with a mutex

#35
post #32

Earlier quoted context omitted.

> I'm not sure your examples of two actors mutually waiting on each other qualify Of course it qualifies, that's a classical text book example of deadlock.

Isn't that a live lock? Can anything really shield you from that? It seems nothing can, in the same way nothing can really shield you from an infinite loop.

> in the same way nothing can really shield you from an infinite loop.

Fwiw that’s possible with dependent typing, the compiler can require a proof of progress (and thus termination).

Re: What could Go wrong with a mutex

#36
post #32

Earlier quoted context omitted.

> I'm not sure your examples of two actors mutually waiting on each other qualify Of course it qualifies, that's a classical text book example of deadlock.

Isn't that a live lock? Can anything really shield you from that? It seems nothing can, in the same way nothing can really shield you from an infinite loop.

> Isn't that a live lock?

If they are blocked it is not a live lock. If they are spinning in a try_read then it would be a live lock.

> Can anything really shield you from that? It seems nothing can, in the same way nothing can really shield you from an infinite loop

Non-Turing Complete languages can be proven to terminate(they do exist, see total functional programming). Similarly I think there are programming paradigms that might be truly deadlock free but my recollection is fuzzy. IIRC writes must never block and reads must handle all blocking conditions with guarded commands. I think you can still livelock (by failing to do something when woken up for example).

Re: What could Go wrong with a mutex

#37
post #18

Earlier quoted context omitted.

Yes, I agree and covered your point regarding safety in literally the sentence before that which you quoted :)

You didn't, really. You mentioned it in passing then completely ignored the issue without actually addressing how to handle the issue of locking in face of go's lack of panic safety, or human errors.

> You didn't, really. You mentioned it in passing

But it was still mentioned ;) The benefits of defer wasn't the point of my post so I'm obviously not going to write a paragraph covering all the safety concerns that defer addresses.

> then completely ignored the issue without actually addressing how to handle the issue of locking in face of go's lack of panic safety, or human errors.

Because that is a separate point. But I'm happy to discuss that if you wish:

Realistically you should put as little code inside the lock as possible and accept the risk that a panic might be generated inside the lock. However if you keep your code minimal and write thorough unit tests (not just testing expected behaviour but also hammering invalid values into the function too) then you can reduce the risk enough that it is lower than the risk of bad UX due to inefficient use of mutexes.

It's not a particularly great answer and relies heavily on developers writing good code and excellent tests, but this falls back to a point I made elsewhere that Go's tooling for managing goroutines is rather poor considering how much Go is credited for making "multi-threaded" code easy. So you're left with having to trust developers to write good code.

Re: What could Go wrong with a mutex

#38
post #8
post #4

People talk about the lack of generics et al as being Go's downfall but personally what I've found to be the biggest annoyance is the very thing Go boasts to be good at: writing bug/race free multi-threaded code. Channels are vastly oversold given their actual suitability and worse still, add their own classes of race conditions (deadlocks) so you often fallback to using mutexes for those, hopefully rare, occasions y…

Is there a language where deadlocks are not possible ?

Yes, Idris. Dependent types in an effect system allow you to prove that deadlock cannot occur, such that a potential deadlock in user code is a compile time error. Edwin Brady has been working on this for decades[1], and Idris 2[2] has finally made it a reality[3].

1. https://www.type-driven.org.uk/edwinb/papers/fi-cbc.pdf (free; 2009)

2. https://livebook.manning.com/book/type-driven-development-wi... (registration wall; 2017)

3. https://research-repository.st-andrews.ac.uk/handle/10023/23... (free; 2021)

Re: What could Go wrong with a mutex

#39
post #31
post #15

Earlier quoted context omitted.

I think deferring unlocks should be more openly described as an anti-pattern. I do appreciate the convenience of defer generally speaking but it has two problems that specifically hinder effective use of mutexes: 1. it allows you to roll up additional code that doesn't need to be inside the mutex, thus keeping other threads waiting longer for an unlock 2. Benchmarks have shown[1][2] that using defer is actually more…

There are several patterns possible which resolve this problem, your third example actually shows that, as it uses a function which only does the container access in a separate function which uses defer - this guarantees that no other code is executed while the lock is held. Alternatively, you could use an anonymous function inside your function which immediately gets called and performs the locking, lookup and defer…

That doesn't address the overhead of defer though (point 2) which can be a problem if your locks are in a well travelled path.

One might argue that if you have a hot path that depends on mutexes then perhaps you need to restructure your code to avoid sharing data (and thus the need for a mutex). And that point would be valid, generally speaking. But there's always that one edge case where it is unavoidable (or even still the cleaner and faster code).

Re: What could Go wrong with a mutex

#40
post #6

The goroutine-inspect tool they used looks really useful. Does anyone know if there's a similar tool for Rust? I've had a few times in the past where I'm trying to figure out why some async application using tokio is hanging and having a similar tool would be very helpful.

I recently researched this after speccing out which language to choose for some data heavy async apps at work - And the answer is right now, unfortunately, no.

Rust is in a bit of a weird place here since as a language it only provides async primitives, and it relies on frameworks bringing their own async runtimes - So the ball is in the Tokio teams' court to provide.

It is nice to use channels and tasks in Tokio - The stalwart Rust compiler catches pitfalls with memory leaks and deadlocks that you could fall into with Go. However, observing what's happening within the Tokio runtime is a far cry from where Go and its profiling tools are at.

https://github.com/tokio-rs/console - here's an attempt to make a TUI where you can watch everything in motion, which is about as close as you can get to an easy UI into the system.

Post reply on HN