Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Fastest Mutexes

justine.lol

Post reply on HN
Print view View original

81–90 of 360 posts

Re: The Fastest Mutexes

#81

Post by tialaramex »

SkiFire13 wrote: post #77
tialaramex wrote: post #10

Earlier quoted context omitted.

Although it does get there eventually, that Rust thread is about Mara's work, which is why it eventually mentions her January 2023 book. The current Rust mutex implementation (which that thread does talk about later) landed earlier this year and although if you're on Linux it's not (much?) different, on Windows and Mac I believe it's new work. That said, Mara's descriptions of the guts of other people's implementatio…

> although if you're on Linux it's not (much?) different AFAIK one reason to switch was that mutexes on Linux and MacOS were not guaranteed to be moveable, so every rust's Mutex had to box the underlying os mutex and was not const-constructible. So this makes a considerable change.

That's the reason for Mara's Mutex. I know, it doesn't seem like five minutes, but Mara's is now the previous version of the Rust Mutex implementation.

std::sync::Mutex::new became const in 1.63, which was over two years ago.

Re: The Fastest Mutexes

#82

Post by amiga386 »

Uehreka wrote: post #68

So on the one hand, all this Cosmo/ape/redbean stuff sounds incredible, and the comments on these articles are usually pretty positive and don’t generally debunk the concepts. But on the other hand, I never hear mention of anyone else using these things (I get that not everyone shares what they’re doing in a big way, but after so many years I’d expect to have seen a couple project writeups talk about them). Every men…

APE works through cunning trickery that might get patched out any day now (and in OpenBSD, it has been).

Most people producing cross-platform software don't want a single executable that runs on every platform, they want a single codebase that works correctly on each platform they support.

With that in mind that respect, languages like go letting you cross compile for all your targets (provided you avoid CGO) is delightful... but the 3-ways-executable magic trick of APE, while really clever, doesn't inspire confidence it'll work forever, and for the most part it doesn't gain you anything. Each platform has their own packaging/signing requirements. You might as well compile a different target for each platform.

Re: The Fastest Mutexes

#83

Post by BryantD »

kccqzy wrote: post #2

> The reason why Cosmopolitan Mutexes are so good is because I used a library called nsync. It only has 371 stars on GitHub, but it was written by a distinguished engineer at Google called Mike Burrows. Indeed this is the first time I've heard of nsync, but Mike Burrows also wrote Google's production mutex implementation at https://github.com/abseil/abseil-cpp/blob/master/absl/synchr... I'm curious why this mutex imp…

Mike was a legend by the time I got to AV. The myth was that any time the search engine needed to be faster, he came in and rewrote a few core functions and went back to whatever else he was doing. Might be true, I just can't verify it personally. Extremely smart engineer who cares about efficiency.

We did not, however, run on one server for any length of time.

Re: The Fastest Mutexes

#84

Post by another-acct »

almostgotcaught wrote: post #69
jonathrg wrote: post #34

> It's still a new C library and it's a little rough around the edges. But it's getting so good, so fast, that I'm starting to view not using it in production as an abandonment of professional responsibility. What an odd statement. I appreciate the Cosmopolitan project, but these exaggerated claims of superiority are usually a pretty bad red flag.

It's especially a red-flag since an enormous number of projects (almost all of them?) will never tolerate shipping fat binaries (ie what cosmopolitan C is in reality).

Agreed; this is what I've always (silently) thought of those fat binaries. Absolute stroke of genius, no doubt, and also a total abomination (IMO) from a sustainability perspective.

Re: The Fastest Mutexes

#85

Post by PaulDavisThe1st »

pizlonator wrote: post #23

Always cool to see new mutex implementations and shootouts between them, but I don’t like how this one is benchmarked. Looks like a microbenchmark. Most of us who ship fast locks use very large multithreaded programs as our primary way of testing performance. The things that make a mutex fast or slow seem to be different for complex workloads with varied critical section length, varied numbers of threads contending,…

To add to this, as the original/lead author of a desktop app that frequently runs with many tens of threads, I'd like to see numbers on performance in non-heavily contended cases. As a real-time (audio) programmer, I am more concerned with (for example) the cost to take the mutex even when it is not already locked (which is the overwhelming situation in our app). Likewise, I want to know the cost of a try-lock operation that will fail, not what happens when N threads are contending.

Of course, with Cosmopolitan being open source and all, I could do these measurements myself, but still ...

Re: The Fastest Mutexes

#86

Post by another-acct »

tombert wrote: post #4

I have to admit that I have an extremely visceral, negative feeling whenever I see a mutex, simply because I've had to debug enough code written by engineers who don't really know how to use them, so a large part of previous jobs has been to remove locks from code and replace with some kind of queue or messaging abstraction [1]. It's only recently that I've been actively looking into different locking algorithms, jus…

> remove locks from code and replace with some kind of queue or messaging abstraction

Shared-nothing message passing reflects the underlying (modern) computer architecture more closely, so I'd call the above a good move. Shared memory / symmetric multiprocessing is an abstraction that leaks like a sieve; it no longer reflects how modern computers are built (multiple levels of CPU caches, cores, sockets, NUMA, etc).

Re: The Fastest Mutexes

#87

Post by foobazgt »

hinkley wrote: post #50
cogman10 wrote: post #31

Earlier quoted context omitted.

When, how, and why. The biggest part of mutexes and how to properly use them is thinking of the consistency of the data that you are working with. Here's a really common bug (psuedocode) if (lock {data.size()} > 0) { value = lock { data.pop() } lock { foo.add(value) } } The issue here is size can change, pop can change, and foo can change in unexpected ways between each of the acquired locks. The right way to write t…

Composing locks is where Java usually blows up. And computeIfAbsent can end up holding the lock for too long if the function is slow.

Composing locks isn't a Java problem - it's a fundamental abstraction problem with locks. This is one of the reasons why you usually reach for higher level abstractions than mutexes.

> And computeIfAbsent can end up holding the lock for too long if the function is slow.

How is this different from any other lock-holding code written anywhere?

Re: The Fastest Mutexes

#88

Post by ataylor284_ »

PaulDavisThe1st wrote: post #85
pizlonator wrote: post #23

Always cool to see new mutex implementations and shootouts between them, but I don’t like how this one is benchmarked. Looks like a microbenchmark. Most of us who ship fast locks use very large multithreaded programs as our primary way of testing performance. The things that make a mutex fast or slow seem to be different for complex workloads with varied critical section length, varied numbers of threads contending,…

To add to this, as the original/lead author of a desktop app that frequently runs with many tens of threads, I'd like to see numbers on performance in non-heavily contended cases . As a real-time (audio) programmer, I am more concerned with (for example) the cost to take the mutex even when it is not already locked (which is the overwhelming situation in our app). Likewise, I want to know the cost of a try-lock opera…

The writeup suggests this implementation is optimized for the not-locked case:

> uses an optimistic CAS (compare and swap) immediately, so that locking happens quickly when there's no contention

Re: The Fastest Mutexes

#89

Post by 0x1ceb00da »

amiga386 wrote: post #82
Uehreka wrote: post #68

So on the one hand, all this Cosmo/ape/redbean stuff sounds incredible, and the comments on these articles are usually pretty positive and don’t generally debunk the concepts. But on the other hand, I never hear mention of anyone else using these things (I get that not everyone shares what they’re doing in a big way, but after so many years I’d expect to have seen a couple project writeups talk about them). Every men…

APE works through cunning trickery that might get patched out any day now (and in OpenBSD, it has been). Most people producing cross-platform software don't want a single executable that runs on every platform, they want a single codebase that works correctly on each platform they support. With that in mind that respect, languages like go letting you cross compile for all your targets (provided you avoid CGO) is deli…

Wasn't elf format modified by upstream to accomodate for cosmo? That makes it kinda official. Still hard to see a use case for it. If you want everyone to be able to run your program, just write a web app, a win32 program, or a java applet. 20 years old java applets still run on modern JVMs.

Re: The Fastest Mutexes

#90

Post by Dansvidania »

stonethrowaway wrote: post #38

> In 2012, Tunney started working for Google as a software engineer.[4] In March 2014, Tunney petitioned the US government on We the People to hold a referendum asking for support to retire all government employees with full pensions, transfer administrative authority to the technology industry, and appoint the executive chairman of Google Eric Schmidt as CEO of America. the absolute madman

I wonder what they (Tunney) think of that now.
Post reply on HN