Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inside Rust's std and parking_lot mutexes – who wins?

blog.cuongle.dev

Post reply on HN
Print view View original

31–40 of 134 posts

Re: Inside Rust's std and parking_lot mutexes – who wins?

#31

Post by tialaramex »

kccqzy wrote: post #13
wahern wrote: post #5

Earlier quoted context omitted.

> Prior to that issue Rust was using something much worse, pthread_mutex_t Presumably you're referring to this description, from the Github Issue: > > On most platforms, these structures are currently wrappers around their pthread equivalent, such as pthread_mutex_t. These types are not movable, however, forcing us to wrap them in a Box, resulting in an allocation and indirection for our lock types. This also gets in…

I’m actually thinking of the sheer size of pthread mutexes. They are giant. The issue says that they wanted something small, efficient, and const constructible. Pthread mutexes are too large for most applications doing fine-grained locking.

On a typical modern 64-bit Linux for example they're 40 bytes ie they are 320 bits. So yeah, unnecessarily bulky.

On my Linux system today Rust's Mutex> is smaller than the pthread mutex type whether it is locked and has the text "pthread_mutex_t is awful" inside it or maybe unlocked with explicitly no text (not an empty string), either would only take like 30-odd bytes, the pthread_mutex_t is 40 bytes.

On Windows the discrepancy is even bigger, their OS native mutex type is this sprawling 80 byte monster while their Mutex is I believe slightly smaller than on Linux even though it has the same features.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#32

Post by tialaramex »

nemetroid wrote: post #27
wahern wrote: post #5

Earlier quoted context omitted.

> Prior to that issue Rust was using something much worse, pthread_mutex_t Presumably you're referring to this description, from the Github Issue: > > On most platforms, these structures are currently wrappers around their pthread equivalent, such as pthread_mutex_t. These types are not movable, however, forcing us to wrap them in a Box, resulting in an allocation and indirection for our lock types. This also gets in…

> The effect of referring to a copy of the object when locking, unlocking, or destroying it is undefined. https://pubs.opengroup.org/onlinepubs/9699919799/functions/V... I.e., if I pthread_mutex_init(&some_addr, ...), I cannot then copy the bits from some_addr to some_other_addr and then pthread_mutex_lock(&some_other_addr). Hence not movable. > Moving a mutex is otherwise non-sensical once the mutex is visible What…

Well, technically if you only have a mutable borrow (it's not your object) then you can't move from it unless you replace it somehow. If you have two such borrows you can swap them, if the type implements Default you can take from one borrow and this replaces it with its default and if you've some other way to make one you can replace the one you've got a reference to with that one, but if you can't make a new one and don't have one to replace it with, then too bad, no moving the one you've got a reference to.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#34

Post by LegionMammal978 »

kouteiheika wrote: post #16
LegionMammal978 wrote: post #14

Earlier quoted context omitted.

> It's always either an `unwrap` (and we know how well that can go [2]) If a mutex has been poisoned, then something must have already panicked, likely in some other thread, so you're already in trouble at that point. It's fine to panic in a critical section if something's horribly wrong, the problem comes with blindly continuing after a panic in other threads that operate on the same data. In general, you're unlikel…

> If a mutex has been poisoned, then something must have already panicked, likely in some other thread, so you're already in trouble at that point. I find that in the majority of cases you're essentially dealing with one of two cases: 1) Your critical sections are tiny and you know you can't panic, in which case dealing with poisoning is just useless busywork. 2) You use a Mutex to get around Rust's "shared xor mutab…

> 1) Your critical sections are tiny and you know you can't panic, in which case dealing with poisoning is just useless busywork.

Many people underestimate how many things can panic in corner cases. I've found quite a few unsafe functions in various crates that were unsound due to integer-overflow panics that the author hadn't noticed. Knowing for a fact that your operation cannot panic is the exception rather than the rule, and while it's unfortunate that the std Mutex doesn't accomodate non-poisoning mutexes, I see poisoning as a reasonable default.

(If Mutex::lock() unwrapped the error automatically, then very few people would even think about the "useless busywork" of the poison bit. For a similar example, the future types generated for async functions contain panic statements in case they are polled after completion, and no one complains about those.)

> 2) You use a Mutex to get around Rust's "shared xor mutable" requirement. That is, you just want to temporarily grab a mutable reference and modify an object, but you don't have any particular atomicity requirements.

Then I'd stick to a RefCell. Unless it's a static variable in a single-threaded program, in which case I usually just write some short wrapper functions if I find the manipulation too tedious.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#35

Post by jltsiren »

kccqzy wrote: post #19

I will personally recommend that unless you are writing performance sensitive code*, don’t use mutexes at all because they are too low-level an abstraction. Use MPSC queues for example, or something like RCU. I find these abstractions much more developer friendly. *: You may be, since you are using Rust.

A mutex is a natural abstraction when there is exactly one of them. You have a bunch of tasks doing their own stuff, with shared mutable state behind the mutex. When you start thinking about using two mutexes, other abstractions often become more convenient.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#36

Post by nemetroid »

tialaramex wrote: post #32
nemetroid wrote: post #27

Earlier quoted context omitted.

> The effect of referring to a copy of the object when locking, unlocking, or destroying it is undefined. https://pubs.opengroup.org/onlinepubs/9699919799/functions/V... I.e., if I pthread_mutex_init(&some_addr, ...), I cannot then copy the bits from some_addr to some_other_addr and then pthread_mutex_lock(&some_other_addr). Hence not movable. > Moving a mutex is otherwise non-sensical once the mutex is visible What…

Well, technically if you only have a mutable borrow (it's not your object) then you can't move from it unless you replace it somehow. If you have two such borrows you can swap them, if the type implements Default you can take from one borrow and this replaces it with its default and if you've some other way to make one you can replace the one you've got a reference to with that one, but if you can't make a new one an…

You're right and I edited my comment.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#37

Post by scottlamb »

Conscat wrote: post #9
nextaccountic wrote: post #6

Earlier quoted context omitted.

How can a parking_lot lock be less than 1 byte? does this uses unsafe? Rust in general doesn't support bit-level objects unless you cast things to [u8] and do some shifts and masking manually (that is, like C), which of course is wildly unsafe for data structures with safety invariants

This article elaborates how it works.

Unhelpful response. This cuongle.dev article does not answer nextaccountic's question, and neither do the webkit.org articles that describe the parking lot concept but not this Rust implementation. The correct answer appears to be that it's impossible: `parking_lot::RawMutex` has private storage that owns the entire byte and does not provide any accessor for the unused six bits.

https://docs.rs/parking_lot/0.12.5/parking_lot/struct.RawMut...

(unless there's somewhere else in the crate that provides an accessor for this but that'd be a weird interface)

(or you just use transmute to "know" that it's one byte and which bits within the byte it actually cares about, but really don't do that)

(slightly more realistically, you could probably use the `parking_lot_core::park` portion of the implementation and build your own equivalent of `parking_lot::RawMutex` on top of it)

(or you send the `parking_lot` folks a PR to extend `parking_lot::RawMutex` with interface you want; it is open source after all)

Re: Inside Rust's std and parking_lot mutexes – who wins?

#38

Post by sunshowers »

JoshTriplett wrote: post #22
kouteiheika wrote: post #10

> Poisoning: Panic Safety in Mutexes This is one of the biggest design flaws in Rust's std, in my opinion. Poisoning mutexes can have its use, but it's very rare in practice. Usually it's a huge misfeature that only introduces problems. More often than not panicking in a critical section is fine [1], but on the other hand poisoning a Mutex is a very convenient avenue for a denial-of-service attack, since a poisoned M…

We're currently working on separating poison from mutexes, such that the default mutexes won't have poisoning (no more `.lock().unwrap()`), and if you want poisoning you can use something like `Mutex >`.

I'm very disappointed at this. The path of least resistance ought to be the right thing to do.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#39

Post by sunshowers »

kouteiheika wrote: post #17
sunshowers wrote: post #15

Earlier quoted context omitted.

To the contrary, the projects I've been part of have had no end of issues related to being cancelled in the middle of a critical section [1]. I consider poisoning to be table stakes for a mutex. [1] https://sunshowers.io/posts/cancelling-async-rust/#the-pain-...

Well, I mean, if you've made the unfortunate decision to hold a Mutex across await points...? This is completely banned in all of my projects. I have a 100k+ LOC project running in production, that is heavily async and with pervasive usage of threads and mutexes, and I never had a problem, precisely because I never hold a mutex across an await point. Hell, I don't even use async mutexes - I just use normal synchronou…

As I said in the article, we avoid Tokio mutexes entirely for the exact reason that being cancelled in the middle of a critical section is bad. In Rust, there are two sources of cancellations in the middle of a critical section: async cancellations and panics. Ergo, panicking in the middle of a critical section is also bad, and mutexes ought to detect that and mark their internal state as corrupted as a result.

Re: Inside Rust's std and parking_lot mutexes – who wins?

#40

Post by scottlamb »

strbean wrote: post #7
kccqzy wrote: post #3

There was a giant super-long GitHub issue about improving Rust std mutexes a few years back. Prior to that issue Rust was using something much worse, pthread_mutex_t. It explained the main reason why the standard library could not just adopt parking_lot mutexes: From https://github.com/rust-lang/rust/issues/93740 > One of the problems with replacing std's lock implementations by parking_lot is that parking_lot alloca…

Seems like the simple solution to this problem would be to have both, no? A simple native lock in the standard library along with a nicer implementation (also in the standard library) that depends on the simple lock?

The simplest solution is for `std::mutex` to provide a simple, efficient mutex which is a good choice for almost any program. And it does. Niche programs can pull in a crate.

I doubt `parking_lot` would have been broadly used—maybe wouldn't even have been written—if `std` had this implementation from the start.

What specifically in this comparison made you think that `parking_lot` is broadly needed? They had to work pretty hard to find a scenario in which `parking_lot` did much better in any performance metrics. And as I alluded to in another comment, `parking_lot::Mutex` doesn't have a size advantage over `std::mutex::Mutex` when `InnerFoo` has word alignment. That's the most common situation, I think.

If I were to make a wishlist of features for `std::mutex` to just have, it wouldn't be anything `parking_lot` offers. It'd be stuff like the lock contention monitoring that the (C++) `absl::Mutex` has. (And at least on some platforms you can do a decent job of monitoring this with `std::mutex` by monitoring the underlying futex activity.)

Post reply on HN