Live data from Hacker News

The Deadlock Empire: An Interactive Guide to Locks

deadlockempire.github.io

31–40 of 44 posts

Re: The Deadlock Empire: An Interactive Guide to Locks

#31

It's missing my favourite optimistic lock, the Delusional Lock.

I'm assuming it's the "I'm sure it'll be fine to read/write this unsynchronized, what's the worst that could happen?" locking strategy.

"what's the worst that could happen?" implies some forethought. Delusional Locking just goes "It's gonna be fine! LALALALALA I CaN'T HeAr YoU"

Re: The Deadlock Empire: An Interactive Guide to Locks

#32
post #26

Earlier quoted context omitted.

In the mainstream languages, synchronisation seems to be a means (locking code) to an end (locking data). The programmer usually does not care that only one thread can enter a method at a time - that's just the 'how'. The 'why' is that the programmer wants to reason about reads and writes in a multithreaded environment as easily as one would do in a single-threaded environment. Compare lock management to memory manag…

Channel-based programming, perhaps? From Go style to Erlang shared-nothing, those approaches offer some of the properties you’re looking for. Or maybe the holy grail hasn’t been created yet in this category: a compiler that can transform arbitrary computations or expressions of computational goals into their maximally parallel form, where locks etc. are compiler output artifacts a la assembly instructions rather than…

What do you think of this https://github.com/HigherOrderCO/Bend ?

Re: The Deadlock Empire: An Interactive Guide to Locks

#33
What this really explains is why you should use Python. I use ThreadPoolExecutor + as_completed on every I/O-bound or async workflow reflexively and have literally never encountered a deadlock. (With this method, I mean. I have encountered, and caused, many deadlocks in less pristine environments.)

Re: The Deadlock Empire: An Interactive Guide to Locks

#34

Earlier quoted context omitted.

Rust's borrow checker will of course check that if you borrowed the protected item you've given it back before unlocking, whereas all Folly and similar C++ libraries can do here is caution you that this footgun exists and is loaded and pointed at your foot so please don't.

That's just the usual resource ownership management problem that Rust is supposed to solve. But a simple templated type like GP proposed does indeed fix the issue discussed here. To access the thing in the first place you need to lock the correct mutex. Looking at Folly::Synchronized, locking doesn't even return the protected item itself directly. In most cases -- unless the bare pointer is needed -- you will access…

GP isn't totally wrong. With folly::Synchronized, you can lock, take a reference, then unlock, and continue to (incorrectly/unsafely) use the reference. The compiler doesn't catch that.

  folly::Synchronized lockedObj;
  auto lockHandle = lockedObj.wlock();
  auto& myReference = *lockHandle;
  lockHandle.unlock();
  myReference = 5; // bad
Still, it is harder to misuse than bare locks unattached to data.

Re: The Deadlock Empire: An Interactive Guide to Locks

#35
post #15

Earlier quoted context omitted.

> The big win in Rust is that the mutexes are tied to the data they protect. The compiler won't let you access data until it's locked. You can get this kind of behavior in C++ with, e.g., folly::Synchronized .

Rust's borrow checker will of course check that if you borrowed the protected item you've given it back before unlocking, whereas all Folly and similar C++ libraries can do here is caution you that this footgun exists and is loaded and pointed at your foot so please don't.

Yes.

Re: The Deadlock Empire: An Interactive Guide to Locks

#36
Piggy backing on this post to ask, does anyone know of good multithreaded testing libraries akin to what thread weaver used to be before it got abandoned?

https://github.com/google/thread-weaver

Happy to discuss frameworks for other languages too as I love using it as a tool to explore different possible ways that concurrency can go wrong even if you might never encounter them in standard testing.

Re: The Deadlock Empire: An Interactive Guide to Locks

#37
post #34

Earlier quoted context omitted.

That's just the usual resource ownership management problem that Rust is supposed to solve. But a simple templated type like GP proposed does indeed fix the issue discussed here. To access the thing in the first place you need to lock the correct mutex. Looking at Folly::Synchronized, locking doesn't even return the protected item itself directly. In most cases -- unless the bare pointer is needed -- you will access…

GP isn't totally wrong. With folly::Synchronized, you can lock, take a reference, then unlock, and continue to (incorrectly/unsafely) use the reference. The compiler doesn't catch that. folly::Synchronized lockedObj; auto lockHandle = lockedObj.wlock(); auto& myReference = *lockHandle; lockHandle.unlock(); myReference = 5; // bad Still, it is harder to misuse than bare locks unattached to data.

Yes, but you can also take a reference (copy a pointer), delete, and continue to use the reference etc. I was pointing out that this is simply a lifetime/ownership issue, not an issue specific to locking (and yes, Rust solves that issue, at least for the easy cases). And as far as the problem is protecting access to a locked resource, a class like folly::Synchronized does indeed solve the problem.

Re: The Deadlock Empire: An Interactive Guide to Locks

#38

Earlier quoted context omitted.

I'm assuming it's the "I'm sure it'll be fine to read/write this unsynchronized, what's the worst that could happen?" locking strategy.

That just sounds like the "read uncommitted" transaction level. Works as long as you know what to do :)

IIRC some IBM DBs have a level called 'chaos', it may be read uncommitted but with other side effects...

Re: The Deadlock Empire: An Interactive Guide to Locks

#39
post #8

Earlier quoted context omitted.

I'll give you one tip that covers a large portion of deadlocks I've seen in my career. A lock should be used to lock a data structure, that's it. Corollary: Never hold a lock while calling a function. Most common deadlocking scenario is people taking a lock, calling a function, that they don't know takes a different lock. If you can't get by just taking a lock, touching some data structure (lightly), and releasing th…

That sounds good in principle but is it practical? As long as you have something to do while holding the lock, chances are that implementing that something requires calling a function. That or code duplication.

In my experience it is practical. Let's say you have a shared linked list, you take the lock in your insert function, you insert an item, you give the lock back. No function calls.

Code that looks like what you describe, "implementing something that requires calling a function", tends to deadlock or be wrong. A really smart guy I worked with wrote some database driver that looked like that, it worked, except when it deadlocked, and finding that deadlock was a nightmare. I'm sure there are exceptions but this rule will get you out of a lot of trouble. If you need to violate the rule try find a different synchronization/concurrency mechanism or a different data structure.

Even if the code is initially correct, inevitably someone will refactor it without realizing a lock is taken and break it.

Re: The Deadlock Empire: An Interactive Guide to Locks

#40

This thing is great. The deeper lesson seems to be not to try any of these things in any kind of complicated program, you will fuck it up. Imagine writing something with dozens of threads, each running different programs with various threading primitives sprinkled around. You would never be able to debug it, because just conjuring each case of "if this thread gets to here and that thread gets to there" in a test woul…

I don't have many iron rules, but I have one that I refuse to bend on: You've either proved your bespoke multi-threaded code mathematically, or it is guaranteed to be Wrong with a capital W.

If you've proved it correct, you now have just a 50% chance of it being merely wrong when deployed on actual hardware.

Post reply on HN