Mutexes are faster than Spinlocks
matklad.github.io
Mutexes are faster than Spinlocks
1–10 of 150 posts
Re: Mutexes are faster than Spinlocks
#2Not an expert here.
In a spin lock, the lock state is checked in a tight loop by all waiters. This will be using some sort of memory fence. FWIK, memory fence or barriers flush the CPU cache and would initiate reading the variable (spin lock state) for evaluation. I would expect spin locking overheads to increase with number of cores.
On NUMA, I think flushing is more expensive. Hence, spin locks have an additional overhead of having to load and evaluate on every spin as against being woken up for mutexes (like a callback)
Re: Mutexes are faster than Spinlocks
#3couldn't you eliminate the bad spinlock behavior by coding them to be go into an efficient wait if to much spinning is going on ?
Re: Mutexes are faster than Spinlocks
#4Not an expert here. In a spin lock, the lock state is checked in a tight loop by all waiters. This will be using some sort of memory fence. FWIK, memory fence or barriers flush the CPU cache and would initiate reading the variable (spin lock state) for evaluation. I would expect spin locking overheads to increase with number of cores. On NUMA, I think flushing is more expensive. Hence, spin locks have an additional o…
It doesn’t flush the entire cache (that would be a disaster) but it does shoot down the cache line containing the lock in all cores other than the one that acquired the lock.
The real issue with spin locks is fairness. There’s no assurance that any given thread will ever make progress. A thread could starve forever. Production-ready mutexes like absl::Mutex make efforts toward fairness, even if they don’t have hard guarantees.
Re: Mutexes are faster than Spinlocks
#5couldn't you eliminate the bad spinlock behavior by coding them to be go into an efficient wait if to much spinning is going on ?
That’s what virtually all battle-hardened lock libraries do: spin for a bit (but not too tightly, using a pause in the loop) then fall back to waiter lists and futex.
Re: Mutexes are faster than Spinlocks
#6couldn't you eliminate the bad spinlock behavior by coding them to be go into an efficient wait if to much spinning is going on ?
Yes. And then it becomes an adaptive mutex.
Re: Mutexes are faster than Spinlocks
#7Not an expert here. In a spin lock, the lock state is checked in a tight loop by all waiters. This will be using some sort of memory fence. FWIK, memory fence or barriers flush the CPU cache and would initiate reading the variable (spin lock state) for evaluation. I would expect spin locking overheads to increase with number of cores. On NUMA, I think flushing is more expensive. Hence, spin locks have an additional o…
Yes, spin locks require atomic memory operations - as do many other OS primitives. CPUs have dedicated instructions to do this more quickly for locks than generalized coherent memory, for example cmpxchg on x86. There are microarchitectural optimizations to make these things work well.
I've been writing MP code since the early 90's; SPARC, MIPS, x86, PPC, SH-4, Alpha, and all have instructions to make this easier for you.
Spin locks are very useful for locks held for short durations, and only on MP systems, particularly in the kernel, since a spin lock there can deadlock the whole system on a single CPU machine. The very general rule of thumb is that you generally want blocking locks, unless there is a very specific reason to use a spin lock. Those very specific reasons might be waiting briefly on some device in a driver, or running a realtime system where wake latency is critical, etc. Most of the time, spin locks will burn CPU for no good reason, and you still have to implement the non-spinlock case anyway, in case you are running on a 1 CPU system. So, might as well start with blocking locks, and only add spin locks where performance would benefit from it.
Re: Mutexes are faster than Spinlocks
#8couldn't you eliminate the bad spinlock behavior by coding them to be go into an efficient wait if to much spinning is going on ?
Yes! That's called an optimistic spinning lock!
https://linuxplumbersconf.org/event/4/contributions/286/atta...
Re: Mutexes are faster than Spinlocks
#9Not an expert here. In a spin lock, the lock state is checked in a tight loop by all waiters. This will be using some sort of memory fence. FWIK, memory fence or barriers flush the CPU cache and would initiate reading the variable (spin lock state) for evaluation. I would expect spin locking overheads to increase with number of cores. On NUMA, I think flushing is more expensive. Hence, spin locks have an additional o…
> checked in a tight loop by all waiters
This actually does not have to be this way. You could have a linked list of spinlocks, one for each waiter. Each waiter spins on its own, unique spinlock. When the previous waiter is done it unlocks the next spinlock, and so on. The implementation gets a bit complicated on non-GC languages, since there are races between insertion/removal on the linked list. If the number of threads is fixed and long-lived then it becomes easier, since instead of a linked list you can have an array of spinlocks.
Note: in some architectures (x86?) you could possibly do away with atomics, since (I believe) int updates are atomic by default. Not really sure though.
Re: Mutexes are faster than Spinlocks
#10Not an expert here. In a spin lock, the lock state is checked in a tight loop by all waiters. This will be using some sort of memory fence. FWIK, memory fence or barriers flush the CPU cache and would initiate reading the variable (spin lock state) for evaluation. I would expect spin locking overheads to increase with number of cores. On NUMA, I think flushing is more expensive. Hence, spin locks have an additional o…
It doesn’t flush the entire cache (that would be a disaster) but it does shoot down the cache line containing the lock in all cores other than the one that acquired the lock. The real issue with spin locks is fairness. There’s no assurance that any given thread will ever make progress. A thread could starve forever. Production-ready mutexes like absl::Mutex make efforts toward fairness, even if they don’t have hard g…
Aren't most modern spinlock implementations are ticket-based, which ensures fairness among waiters (FIFO-like)? The linux kernel implementations most definitely are.
I agree that the naive implementations are not.