Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inside Rust's std and parking_lot mutexes – who wins?

blog.cuongle.dev

Post reply on HN
Print view View original

91–100 of 134 posts

Re: Inside Rust's std and parking_lot mutexes – who wins?

#91

Post by dap »

JoshTriplett wrote: post #57
sunshowers wrote: post #38

Earlier quoted context omitted.

I'm very disappointed at this. The path of least resistance ought to be the right thing to do.

In the entire history of the standard library, we have never once seen a single report of anyone attempting to recover from poison.

That’s not surprising to me, but it’s not much of an argument for changing the default to be less safe. Most people want poisoning to propagate fatal errors and avoid reading corrupted data, not to recover from panics.

Edit: isn’t that an argument not to change the default? If people were recovering from poison a lot and that was painful, that’s one thing. But if people aren’t doing that, why is this a problem?

Re: Inside Rust's std and parking_lot mutexes – who wins?

#92

Post by scottlamb »

pizlonator wrote: post #67
scottlamb wrote: post #37

Earlier quoted context omitted.

Unhelpful response. This cuongle.dev article does not answer nextaccountic's question, and neither do the webkit.org articles that describe the parking lot concept but not this Rust implementation. The correct answer appears to be that it's impossible: `parking_lot::RawMutex` has private storage that owns the entire byte and does not provide any accessor for the unused six bits. https://docs.rs/parking_lot/0.12.5/par…

> and neither do the webkit.org articles that describe the parking lot concept but not this Rust implementation The WebKit post explicitly talks about how you just need two bits to describe the lock state. > The correct answer appears to be that it's impossible: `parking_lot::RawMutex` has private storage that owns the entire byte and does not provide any accessor for the unused six bits. Not impossible. One way to d…

> The WebKit post explicitly talks about how you just need two bits to describe the lock state.

It describes the algorithm but not how a caller of the Rust `parking_lot` crate could take advantage of this.

> Not impossible. One way to do this is to just use parking_lot directly.

By "just use parking_lot directly", I think you're talking about reimplementing the parking lot algorithm or using the C++ `WTF::ParkingLot` implementation? But not actually using the existing Rust crate called `parking_lot` described in the cuongle.dev article? That's confusingly put, and nextaccountic's question is certainly Rust-specific and likely expecting an answer relating to this particular crate. At the least, "does this use unsafe" would certainly be true with an implementation from scratch or when using FFI into C++.

I hear that this algorithm and the C++ implementation are your invention, and all due respect for that. I'm also hearing that you are not familiar with this Rust implementation. It does not offer the main benefit you're describing. `parking_lot::RawMutex` is a one-byte type; that six bits within it are unused is true but something callers can not take advantage of. Worse, `parking_lot::Mutex` in practice is often a full word larger than `InnerFoo` due to alignment padding. As such, there's little benefit over a simpler futex-based approach.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#93

Post by JoshTriplett »

questioner8216 wrote: post #71
JoshTriplett wrote: post #69

Earlier quoted context omitted.

Speaking only for myself (though several other people have expressed the same sentiment), I wish we could get rid of unwinding. That would be a massive challenge to do while preserving capabilities people care about, such as the ability to handle panics in http request handlers without exiting. I think it would be possible, though.

That sounds really interesting, whether it is done in Rust, some Rust 2.0, or a successor or experimental language. I do not know whether it is possible, though. If one does not unwind, what should actually happen instead? How would for instance partial computations, and resources on the stack, be handled? Some partial or constrained unwinding? I have not given it a lot of thought, though. How do languages without ex…

Vague sketch: imagine if we had scoped panic hooks, unhooked via RAII. So, for use cases that today use unwinding for cleanup (e.g. "switch the terminal back out of curses mode"), you do that cleanup in a panic hook instead.

The hard use case to handle without unwinding is an HTTP server that wants to allow for panics in a request handler without panicking the entire process. Unwinding is a janky way to handle that, and creates issues in code that doesn't expect unwinding (e.g. half-modified states), and poisoning in particular seems likely to cascade and bring down other parts of the process anyway if some needed resource gets poisoned. But we need a reasonable alternative to propose for that use case, in order to seriously evaluate eliminating unwinding.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#94

Post by questioner8216 »

JoshTriplett wrote: post #93
questioner8216 wrote: post #71

Earlier quoted context omitted.

That sounds really interesting, whether it is done in Rust, some Rust 2.0, or a successor or experimental language. I do not know whether it is possible, though. If one does not unwind, what should actually happen instead? How would for instance partial computations, and resources on the stack, be handled? Some partial or constrained unwinding? I have not given it a lot of thought, though. How do languages without ex…

Vague sketch: imagine if we had scoped panic hooks, unhooked via RAII. So, for use cases that today use unwinding for cleanup (e.g. "switch the terminal back out of curses mode"), you do that cleanup in a panic hook instead. The hard use case to handle without unwinding is an HTTP server that wants to allow for panics in a request handler without panicking the entire process. Unwinding is a janky way to handle that,…

I am not sure that I understand what scoped panic hooks would or might look like. Are they maybe similar to something like try-catch-finally in Java? Would the language force the programmer to include them in certain cases somehow?

If a request handler for example has at some point in time 7 nested calls, in call no. 2 and call no. 6 have resources and partial computation that needs clean-up somehow and somewhere, and call no. 7 panics, I wonder what the code would look like in the different calls, and what would happen and when, and what the compiler would require, and what other relevant code would look like.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#95

Post by sunshowers »

exDM69 wrote: post #81
sunshowers wrote: post #15

Earlier quoted context omitted.

To the contrary, the projects I've been part of have had no end of issues related to being cancelled in the middle of a critical section [1]. I consider poisoning to be table stakes for a mutex. [1] https://sunshowers.io/posts/cancelling-async-rust/#the-pain-...

Worth noting that this is not `std::mutex` or `parking_lot::mutex` as discussed in the article, but `tokio::sync::Mutex` in cancellable async code.

Correct yes, but my point was about cancellation in the middle of a critical section more generally.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#96

Post by sunshowers »

simonask wrote: post #77
sunshowers wrote: post #61

Earlier quoted context omitted.

Oh, I don't think recovery from poison is why poisoning is good. The reason poisoning is good is that at the moment you've acquired a lock on a mutex, you should be able to assume that the invariants guarded by the mutex are upheld (and panic if not).

Mutex doesn't promise to uphold any more invariants than `&mut T` does. If the state can be corrupted by a panic while holding `&mut T`, I don't think there's any good reason to expect that obtaining it through `MutexGuard` should make any difference. Panic propagation is typically handled much better at thread `join()` boundaries.

I understand what you mean, but you're saying has not been true for me in practice. Mutexes absolutely are used to uphold invariants in a way that &mut T is much less often.

There's something to be said here about what I've sometimes called the cancellation blast radius. The issues with cancellation happen when the data corruption/invariant violation is externally visible (if the corrupt data is torn down, who cares.) Mutexes make data corruption externally visible very often.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#97

Post by fpoling »

kccqzy wrote: post #90
fpoling wrote: post #42

Earlier quoted context omitted.

I have found out that mutex solutions are more maintainable and amendable without big redesigns compared with channels or RCU. Consider a simple case of single producer-single consumer. While one can use bounded channels to implement back-pressure, in practice when one wants to either drop messages or apply back-pressure based on message priority any solution involving channels will lead to pile of complex multi-chan…

A channel can be backed by a priority queue if you wish. It’s just an abstraction. The channel internally probably uses mutexes too; it’s just that it’s helpful not to see mutexes in application code.

Surely one can abstract priority queue with mutexes into own data structure. However it will contain enough application-specific logic so a chance of reuse will be slim. So by directly working with mutexes one will have simpler code overall that can still be more easy to adapt to changing requirements.

In general the problem with channels is that they are not flexible enough while being rather abstract. A better abstraction is message passing with one message queue per thread like in Erlang. IMO it can cover more cases before one needs mutexes. But even with that proper back pressure and rate limiting is hard.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#98

Post by simonask »

sunshowers wrote: post #96
simonask wrote: post #77

Earlier quoted context omitted.

Mutex doesn't promise to uphold any more invariants than `&mut T` does. If the state can be corrupted by a panic while holding `&mut T`, I don't think there's any good reason to expect that obtaining it through `MutexGuard` should make any difference. Panic propagation is typically handled much better at thread `join()` boundaries.

I understand what you mean, but you're saying has not been true for me in practice. Mutexes absolutely are used to uphold invariants in a way that &mut T is much less often. There's something to be said here about what I've sometimes called the cancellation blast radius. The issues with cancellation happen when the data corruption/invariant violation is externally visible (if the corrupt data is torn down, who cares.…

In projects I've worked on, this just hasn't been the case. Mutexes, especially in Rust, can grant you a `&mut T` when what you have is `&Mutex`, and that's it - failing to uphold invariants in the API surface of `T` is a bug whether or not it lives inside a mutex.

Lots of data structures need to care about panic-safety. Inserting a node in a tree must leave the tree in a valid state if allocating memory for the new node fails, for example. All of that is completely orthogonal to whether or not the data structure is also observable from multiple threads behind a mutex, and I would argue especially in the case of mutex, whose purpose it is to make an object usable from multiple threads as-if they had ownership.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#99

Post by spacechild1 »

tialaramex wrote: post #31
kccqzy wrote: post #13

Earlier quoted context omitted.

I’m actually thinking of the sheer size of pthread mutexes. They are giant. The issue says that they wanted something small, efficient, and const constructible. Pthread mutexes are too large for most applications doing fine-grained locking.

On a typical modern 64-bit Linux for example they're 40 bytes ie they are 320 bits. So yeah, unnecessarily bulky. On my Linux system today Rust's Mutex > is smaller than the pthread mutex type whether it is locked and has the text "pthread_mutex_t is awful" inside it or maybe unlocked with explicitly no text (not an empty string), either would only take like 30-odd bytes, the pthread_mutex_t is 40 bytes. On Windows t…

> On Windows the discrepancy is even bigger, their OS native mutex type is this sprawling 80 byte monster

I guess you are referring to CRITICAL_SECTION? SRWLock, which has the size of a pointer, has been introduced in Windows Vista. Since Windows 8 you can use WaitOnAddress to build even smaller locks.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#100

Post by tialaramex »

pxeger1 wrote: post #89
tialaramex wrote: post #24

Earlier quoted context omitted.

> This means SRW locks on Windows, and futex-based locks on Linux, some BSDs, and Wasm. Note that the SRW Locks are gone, except if you're on a very old Windows. So today the Rust built-in std mutex for your platform is almost certainly basically a futex though if it is on Windows it is not called a futex and from some angles is better - the same core ideas of the futex apply, we only ask the OS to do any work when w…

> if it is on Windows it is not called a futex What is it called?

WaitOnAddress or from Rust's point of view wait_on_address
Post reply on HN