Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inside Rust's std and parking_lot mutexes – who wins?

blog.cuongle.dev

Post reply on HN
Print view View original

21–30 of 134 posts

Re: Inside Rust's std and parking_lot mutexes – who wins?

#21

Post by pizlonator »

nextaccountic wrote: post #6
pizlonator wrote: post #2

Author of the original WTF::ParkingLot here (what rust’s parking_lot is based on). I’m surprised that this only compared to std on one platform (Linux). The main benefit of parking lot is that it makes locks very small, which then encourages the use of fine grained locking. For example, in JavaScriptCore (ParkingLot’s first customer), we stuff a 2-bit lock into every object header - so if there is ever a need to do s…

How can a parking_lot lock be less than 1 byte? does this uses unsafe? Rust in general doesn't support bit-level objects unless you cast things to [u8] and do some shifts and masking manually (that is, like C), which of course is wildly unsafe for data structures with safety invariants

Original post: https://webkit.org/blog/6161/locking-in-webkit/

Post that mentions the two bit lock: https://webkit.org/blog/7122/introducing-riptide-webkits-ret...

I don’t know the details of the Rust port but I don’t imagine the part that involves the two bits to require unsafe, other than in the ways that any locking algorithm dances with unsafety in Rust (ownership relies on locking algorithms being correct)

Re: Inside Rust's std and parking_lot mutexes – who wins?

#22

Post by JoshTriplett »

kouteiheika wrote: post #10

> Poisoning: Panic Safety in Mutexes This is one of the biggest design flaws in Rust's std, in my opinion. Poisoning mutexes can have its use, but it's very rare in practice. Usually it's a huge misfeature that only introduces problems. More often than not panicking in a critical section is fine [1], but on the other hand poisoning a Mutex is a very convenient avenue for a denial-of-service attack, since a poisoned M…

We're currently working on separating poison from mutexes, such that the default mutexes won't have poisoning (no more `.lock().unwrap()`), and if you want poisoning you can use something like `Mutex>`.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#24

Post by tialaramex »

kccqzy wrote: post #3

There was a giant super-long GitHub issue about improving Rust std mutexes a few years back. Prior to that issue Rust was using something much worse, pthread_mutex_t. It explained the main reason why the standard library could not just adopt parking_lot mutexes: From https://github.com/rust-lang/rust/issues/93740 > One of the problems with replacing std's lock implementations by parking_lot is that parking_lot alloca…

> This means SRW locks on Windows, and futex-based locks on Linux, some BSDs, and Wasm.

Note that the SRW Locks are gone, except if you're on a very old Windows. So today the Rust built-in std mutex for your platform is almost certainly basically a futex though if it is on Windows it is not called a futex and from some angles is better - the same core ideas of the futex apply, we only ask the OS to do any work when we're contended, there is no OS limited resource (other than memory) and our uncontended operations are as fast as they could ever be.

SRW Locks were problematic because they're bulkier than a futex (though mostly when contended) and they have a subtle bug and for a long time it was unclear when Microsoft would get around to fixing that which isn't a huge plus sign for an important intrinsic used in all the high performance software on a $$$ commercial OS...

Mara's work (which you linked) is probably more work, and more important, but it's not actually the most recent large reworking of Rust's Mutex implementation.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#25

Post by writebetterc »

pizlonator wrote: post #21
nextaccountic wrote: post #6

Earlier quoted context omitted.

How can a parking_lot lock be less than 1 byte? does this uses unsafe? Rust in general doesn't support bit-level objects unless you cast things to [u8] and do some shifts and masking manually (that is, like C), which of course is wildly unsafe for data structures with safety invariants

Original post: https://webkit.org/blog/6161/locking-in-webkit/ Post that mentions the two bit lock: https://webkit.org/blog/7122/introducing-riptide-webkits-ret... I don’t know the details of the Rust port but I don’t imagine the part that involves the two bits to require unsafe, other than in the ways that any locking algorithm dances with unsafety in Rust (ownership relies on locking algorithms being correct)

This is very similar to how Java's object monitors are implemented. In OpenJDK, the markWord uses two bits to describe the state of an Object's monitor (see markWord.hpp:55). On contention, the monitor is said to become inflated, which basically means revving up a heavier lock and knowing how to find it.

I'm a bit disappointed though, I assumed that you had a way of only using 2 bits of an object's memory somehow, but it seems like the lock takes a full byte?

Re: Inside Rust's std and parking_lot mutexes – who wins?

#26

Post by loeg »

strbean wrote: post #7
kccqzy wrote: post #3

There was a giant super-long GitHub issue about improving Rust std mutexes a few years back. Prior to that issue Rust was using something much worse, pthread_mutex_t. It explained the main reason why the standard library could not just adopt parking_lot mutexes: From https://github.com/rust-lang/rust/issues/93740 > One of the problems with replacing std's lock implementations by parking_lot is that parking_lot alloca…

Seems like the simple solution to this problem would be to have both, no? A simple native lock in the standard library along with a nicer implementation (also in the standard library) that depends on the simple lock?

My takeaway is that the documentation should make more explicit recommendations depending on the situation -- i.e., people writing custom allocators should use std mutexes; most libraries and allocations that are ok with allocation should use parking_lot mutexes; embedded or libraries that don't want to depend on allocate should use std mutexes. Or maybe parking_lot is almost useless unless you're doing very fine-grained locking. Something like that.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#27

Post by nemetroid »

wahern wrote: post #5
kccqzy wrote: post #3

There was a giant super-long GitHub issue about improving Rust std mutexes a few years back. Prior to that issue Rust was using something much worse, pthread_mutex_t. It explained the main reason why the standard library could not just adopt parking_lot mutexes: From https://github.com/rust-lang/rust/issues/93740 > One of the problems with replacing std's lock implementations by parking_lot is that parking_lot alloca…

> Prior to that issue Rust was using something much worse, pthread_mutex_t Presumably you're referring to this description, from the Github Issue: > > On most platforms, these structures are currently wrappers around their pthread equivalent, such as pthread_mutex_t. These types are not movable, however, forcing us to wrap them in a Box, resulting in an allocation and indirection for our lock types. This also gets in…

> The effect of referring to a copy of the object when locking, unlocking, or destroying it is undefined.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/V...

I.e., if I pthread_mutex_init(&some_addr, ...), I cannot then copy the bits from some_addr to some_other_addr and then pthread_mutex_lock(&some_other_addr). Hence not movable.

> Moving a mutex is otherwise non-sensical once the mutex is visible

What does "visible" mean here? In Rust, in any circumstance where a move is possible, there are no other references to that object, hence it is safe to move.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#28

Post by kouteiheika »

JoshTriplett wrote: post #22
kouteiheika wrote: post #10

> Poisoning: Panic Safety in Mutexes This is one of the biggest design flaws in Rust's std, in my opinion. Poisoning mutexes can have its use, but it's very rare in practice. Usually it's a huge misfeature that only introduces problems. More often than not panicking in a critical section is fine [1], but on the other hand poisoning a Mutex is a very convenient avenue for a denial-of-service attack, since a poisoned M…

We're currently working on separating poison from mutexes, such that the default mutexes won't have poisoning (no more `.lock().unwrap()`), and if you want poisoning you can use something like `Mutex >`.

Yeah, I'm looking forward to it!

While we're at it, another thing that'd be nice to get rid of is `AssertUnwindSafe`, which I find even more pointless.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#29

Post by zozbot234 »

writebetterc wrote: post #25
pizlonator wrote: post #21

Earlier quoted context omitted.

Original post: https://webkit.org/blog/6161/locking-in-webkit/ Post that mentions the two bit lock: https://webkit.org/blog/7122/introducing-riptide-webkits-ret... I don’t know the details of the Rust port but I don’t imagine the part that involves the two bits to require unsafe, other than in the ways that any locking algorithm dances with unsafety in Rust (ownership relies on locking algorithms being correct)

This is very similar to how Java's object monitors are implemented. In OpenJDK, the markWord uses two bits to describe the state of an Object's monitor (see markWord.hpp:55). On contention, the monitor is said to become inflated , which basically means revving up a heavier lock and knowing how to find it. I'm a bit disappointed though, I assumed that you had a way of only using 2 bits of an object's memory somehow, b…

The idea is that six bits in the byte are free to use as you wish. Of course you'll need to implement operations on those six bits as CAS loops (which nonetheless allow for any arbitrary RMW operation) to avoid interfering with the mutex state.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#30

Post by LegionMammal978 »

kprotty wrote: post #18
LegionMammal978 wrote: post #14

Earlier quoted context omitted.

> It's always either an `unwrap` (and we know how well that can go [2]) If a mutex has been poisoned, then something must have already panicked, likely in some other thread, so you're already in trouble at that point. It's fine to panic in a critical section if something's horribly wrong, the problem comes with blindly continuing after a panic in other threads that operate on the same data. In general, you're unlikel…

> so you have no clue if the shared data might be incompletely modified or otherwise logically corrupted. One can make a panic wrapper type if they cared: It's what the stdlib Mutex currently does: MutexGuard checks if its panicking during drop using `std::thread::panicking()`, and if so, sets a bool on the Mutex. The next acquirer checks for that bool & knows state may be corrupted. No need to bake this into the Mut…

My point is that "blindly continuing" is not a great default if you "don't care". If you continue, then you first have to be aware that a multithreaded program can and will continue after a panic in the first place (most people don't think about panics at all), and you also have to know the state of the data after every possible panic, if any. Overall, you have to be quite careful if you want to continue properly, without risking downstream bugs.

The design with a verbose ".lock().unwrap()" and no easy opt-out is unfortunate, but conceptually, I see poisoning as a perfectly acceptable default for people who don't spend all their time musing over panics and their possible causes and effects.

Post reply on HN