Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Fastest Mutexes

justine.lol

Post reply on HN
Print view View original

271–280 of 360 posts

Re: The Fastest Mutexes

#271

Post by karel-3d »

amiga386 wrote: post #82
Uehreka wrote: post #68

So on the one hand, all this Cosmo/ape/redbean stuff sounds incredible, and the comments on these articles are usually pretty positive and don’t generally debunk the concepts. But on the other hand, I never hear mention of anyone else using these things (I get that not everyone shares what they’re doing in a big way, but after so many years I’d expect to have seen a couple project writeups talk about them). Every men…

APE works through cunning trickery that might get patched out any day now (and in OpenBSD, it has been). Most people producing cross-platform software don't want a single executable that runs on every platform, they want a single codebase that works correctly on each platform they support. With that in mind that respect, languages like go letting you cross compile for all your targets (provided you avoid CGO) is deli…

what worked for me for cross-compiling go:

https://github.com/elastic/golang-crossbuild docker images for macos and windows (not linux though)

https://github.com/rust-cross/rust-musl-cross docker images for linux (yes, it's possible to use the same one for go, you just need to put go binary there) so it's static with musl libc and sure to not have some weird issues with old/new libc on old/new/whatever linuxes

(it might be over the top and the zig thing might effectively do the same thing, but I never got it to compile)

oh and apple-codesign rust cargo to get through the signing nonsense

Re: The Fastest Mutexes

#272

Post by xxs »

intelVISA wrote: post #119
pizlonator wrote: post #23

Always cool to see new mutex implementations and shootouts between them, but I don’t like how this one is benchmarked. Looks like a microbenchmark. Most of us who ship fast locks use very large multithreaded programs as our primary way of testing performance. The things that make a mutex fast or slow seem to be different for complex workloads with varied critical section length, varied numbers of threads contending,…

Fast locks are an oxymoron vs optimized CAS

Indeed, b/c locks need to have their CAS.

Re: The Fastest Mutexes

#273

Post by simonask »

SleepyMyroslav wrote: post #265

Completely tangential: As gamedev I came to love slow mutexes that do a lot of debug things in all 'developer' builds. Have debug names/IDs, track owners, report time spent in contention to profiler, report ownership changes to profiler... People tend to structure concurrency differently and games came to some patterns to avoid locks. But they are hard to use and require programmer to restructure things. Most of the…

Even more tangentially: This is one of the reasons I'm having a great time developing a game in Rust.

You never want lock contention in a game if at all avoidable, and in a lot of cases taking a lock is provably unnecessary. For example: Each frame is divided into phases, and mutable access to some shared resource only needs to happen in one specific phase (like the `update()` function before `render()`, or when hot-reloading assets). With scoped threads and Rust's borrowing rules, you can structure things to not even need a mutex, and be completely certain that you will receive a stern compiler error if the code changes in way so you do.

When possible, I'd always love to take a compiler error over a spike in the profiler.

Re: The Fastest Mutexes

#274

Post by xxs »

spacechild1 wrote: post #253
pizlonator wrote: post #106

Earlier quoted context omitted.

Totally! Pro tip: if you really do know that contention is unlikely, and uncontended acquisition is super important, then it's theoretically impossible to do better than a spinlock. Reason: locks that have the ability to put the thread to sleep on a queue must do compare-and-swap (or at least an atomic RMW) on `unlock`. But spinlocks can get away with just doing a store-release (or just a store with a compiler fence…

> If you want your spinlock to be hella fast on contention just make sure you `sched_yield` before each retry Shouldn't you rather use pause instructions (_mm_pause, __yield, ISB, etc.) instead of thread switching? That's what I have seen in most spin lock implementations. Maybe it depends on the use case?

start with few attempts w/o pause, just busy CAS, then CAS + pause, then yield (and backoff/sleep with random duration)

Re: The Fastest Mutexes

#276

Post by pnt12 »

amiga386 wrote: post #58

If it's so good, why haven't all C libraries adopted the same tricks? My betting is that its tricks are only always-faster for certain architectures, or certain CPU models, or certain types of workload / access patterns... and a proper benchmarking of varied workloads on all supported hardware would not show the same benefits. Alternatively, maybe the semantics of the pthread API (that cosmopolitan is meant to be imp…

> If it's so good, why haven't all C libraries adopted the same tricks?

A man and a statiscian are walking down the street when they see a 50€ bill. The statistician keeps walking but the man stops and says "hey, look at this cash on the floor?". But the statistician, uninpressed, says: "must be fake, or someone would have already picked it up". And continues walking. The other man grabs the cash and takes it.

Re: The Fastest Mutexes

#277

Post by int_19h »

jart wrote: post #249
int_19h wrote: post #216

Earlier quoted context omitted.

JVM bytecode is obviously not what I was talking about - it is neither low-level (if you can't efficiently compile the entirety of C into it, it's not good enough for this purpose), nor did it ever have universal first-class OS support. And stuff like https://github.com/jart/cosmopolitan/issues/1263 shows that just because it works today, it doesn't mean that it'll work tomorrow. What's the guarantee that the same wo…

I agree. As a user you should have those guarantees. Here's how I'm helping you get them. - With Windows, APE is just PE and WIN32 so those are already guaranteed by Microsoft. - With Linux, I've been working with kernel maintainers to build consensus around around a patch https://justine.lol/ape.patch that will let your APE binaries be loaded by binfmt_elf. This will provide you with a guarantee that they'll still w…

I rather suspect that Apple will not play along.

That said, a fat binary format that works for both Windows and Linux is already very useful.

Re: The Fastest Mutexes

#278

Post by sublimefire »

blenderob wrote: post #124
Uehreka wrote: post #68

So on the one hand, all this Cosmo/ape/redbean stuff sounds incredible, and the comments on these articles are usually pretty positive and don’t generally debunk the concepts. But on the other hand, I never hear mention of anyone else using these things (I get that not everyone shares what they’re doing in a big way, but after so many years I’d expect to have seen a couple project writeups talk about them). Every men…

> Is there a catch? I am only speaking for myself here. While cosmo and ape do seem very clever, I do not need this type of clever stuff in my work if the ordinary stuff already works fine. Like for example if I can already cross-compile my project to other OSes and platforms or if I've got the infra to build my project for other OSes and platforms, I've no reason to look for a solution that lets me build one binary…

If the executable format changes then it would affect more than just ape/cosmopolitan. All of the old binaries would just stop working.

Re: The Fastest Mutexes

#279

Post by jart »

int_19h wrote: post #277
jart wrote: post #249

Earlier quoted context omitted.

I agree. As a user you should have those guarantees. Here's how I'm helping you get them. - With Windows, APE is just PE and WIN32 so those are already guaranteed by Microsoft. - With Linux, I've been working with kernel maintainers to build consensus around around a patch https://justine.lol/ape.patch that will let your APE binaries be loaded by binfmt_elf. This will provide you with a guarantee that they'll still w…

I rather suspect that Apple will not play along. That said, a fat binary format that works for both Windows and Linux is already very useful.

If we earn the support of Linux and FreeBSD then we might stand of chance. Apple might ask for a better way to secure ape executables, which could be potentially exciting, since it'd be nice to have a vendor neutral solution to code signing.

Re: The Fastest Mutexes

#280

Post by sublimefire »

jkachmar wrote: post #102
Uehreka wrote: post #68

So on the one hand, all this Cosmo/ape/redbean stuff sounds incredible, and the comments on these articles are usually pretty positive and don’t generally debunk the concepts. But on the other hand, I never hear mention of anyone else using these things (I get that not everyone shares what they’re doing in a big way, but after so many years I’d expect to have seen a couple project writeups talk about them). Every men…

reposting my comment from another time this discussion came up: "Cosmopolitan has basically always felt like the interesting sort of technical loophole that makes for a fun blog post which is almost guaranteed to make it to the front page of HN (or similar) purely based in ingenuity & dedication to the bit. as a piece of foundational technology, in the way that `libc` necessarily is, it seems primarily useful for fun…

The problem with this take is that it is not grounded in facts that should determine if the thing is good or bad.

Logically having one thing instead of multiple makes sense as it simplifies the distribution and bytes stored/transmitted. I think the issue here is that it is not time tested. I believe multiple people in various places around the globe think about it and will try to test it. With time this will either bubble up or be stale.

Post reply on HN