Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bug in reader/writer locks in Windows API

old.reddit.com

Post reply on HN
Print view View original

101–110 of 142 posts

Re: Bug in reader/writer locks in Windows API

#101

Post by magicalhippo »

userbinator wrote: post #80
davekilian wrote: post #28

I was wondering how something so basic could go unnoticed for so long. Halfway down the page on OP's link, a user u/rbmm provides a compelling answer: that there are (possibly expected?) cases where a thread trying to acquire the lock in shared mode can accidentally get it in exclusive mode instead. This is due to interleaving of atomic bit test-and-[re]set operations between the (shared mode acquire) thread and the…

If that "rbmm" is the same person as I've seen on other sites, and the characteristic non-native English is a clue that it is, he certainly knows his stuff. threads holding the lock don't wait for each other Unless you're doing nested locking.

> Unless you're doing nested locking.

In my experience, needing to do nested locking is a sign you're up the wrong creek.

I've rewritten some interfaces and implementations from using nested locking to non-nested, and they became easier to use and much faster.

Not saying there's never a place for them, but I avoid nested locking like the plague.

Re: Bug in reader/writer locks in Windows API

#102

Post by layer8 »

amluto wrote: post #23
rwmj wrote: post #17

Earlier quoted context omitted.

If it's any consolation, our company pays for 20,000 GMail licenses and we seem unable to escalate any issue at all to Google, even major issues such as their clearly not working spam filtering, emails being lost, incorrect deduplication of emails, or their random IMAP throttling.

Is there any vendor of actually good email-as-a-service? O365/Exchange/Outlook/Hotmail is a mess. Google has a support problem. Fastmail has an offline email problem. iCloud is not obviously suitable for professional use. What’s left?

Many VPS/web hosters provide email services along with their custom-domain support, e.g. https://www.inmotionhosting.com/support/email/. It’s worthwhile to look into that space, because you are getting dedicated and responsive customer support instead of an anonymous black-box mess.

Re: Bug in reader/writer locks in Windows API

#103

Post by [deleted] »

zozbot234 wrote: post #52
davekilian wrote: post #28

I was wondering how something so basic could go unnoticed for so long. Halfway down the page on OP's link, a user u/rbmm provides a compelling answer: that there are (possibly expected?) cases where a thread trying to acquire the lock in shared mode can accidentally get it in exclusive mode instead. This is due to interleaving of atomic bit test-and-[re]set operations between the (shared mode acquire) thread and the…

Why does it have to use bit test and set and interleave with other threads, though. AIUI you can use a CAS loop to implement any RMW atomically over word-sized (or double word sized, on many platforms) data. That seems like a no-brainer. For comparison, the Rust implementation for lightweight RWLocks on futex-capable *nix platforms is here: https://doc.rust-lang.org/stable/src/std/sys/unix/locks/fute... It sets the "…

[deleted]

Re: Bug in reader/writer locks in Windows API

#104

Post by rbmm »

AndrewStephens wrote: post #22

Subtle bugs in Reader/Writer locks do not surprise me. I worked on an in-house implementation based on Win32 (before C++11 and std::shared_mutex) and my recollection is that although the implementation sounds simple it is exceedingly easy to make subtle mistakes. The experience left me with such a bad feeling for shared locks that I tend to avoid them unless absolutely required. When I last tested std::shared_mutex,…

several years ago i do own implementation of SRW/PushLocks - of course based on original NeillC code.. https://github.com/rbmm/SRW_ALT/tree/main/PushLock-ALT

they have slightly worse performance compared to MS, when high contention, but in test with this OP case - work well. in my test i not found bugs in implementation, but of course can not be sure that they not exist, very complex logic really. nobody test this, however very simply replace SRW calls to my implementation, by macros in h file

Re: Bug in reader/writer locks in Windows API

#105

Post by rbmm »

davekilian wrote: post #28

I was wondering how something so basic could go unnoticed for so long. Halfway down the page on OP's link, a user u/rbmm provides a compelling answer: that there are (possibly expected?) cases where a thread trying to acquire the lock in shared mode can accidentally get it in exclusive mode instead. This is due to interleaving of atomic bit test-and-[re]set operations between the (shared mode acquire) thread and the…

https://github.com/rbmm/SRW-2 - possible better repro code . where i guarantee repro this without hundreds loops. i be say that "question" in RtlReleaseSRWLockExclusive implementation. it first remove Lock bit from SRW (and after this any thread can enter to lock) and then call RtlpWakeSRWLock. but this 2 operations not atomic. if in the middle (after Lock Bit removed and before RtlpWakeSRWLock executed) another thread acquire shared access - it got exclusive. and RtlpWakeSRWLock also "fail" after this - not wake any waiters. but when shared/exlusive owner then release lock..RtlpWakeSRWLock will be called again and self job. i be of course use bit another implementation, link to which i paste in another comment here

Re: Bug in reader/writer locks in Windows API

#106

Post by rbmm »

userbinator wrote: post #80
davekilian wrote: post #28

I was wondering how something so basic could go unnoticed for so long. Halfway down the page on OP's link, a user u/rbmm provides a compelling answer: that there are (possibly expected?) cases where a thread trying to acquire the lock in shared mode can accidentally get it in exclusive mode instead. This is due to interleaving of atomic bit test-and-[re]set operations between the (shared mode acquire) thread and the…

If that "rbmm" is the same person as I've seen on other sites, and the characteristic non-native English is a clue that it is, he certainly knows his stuff. threads holding the lock don't wait for each other Unless you're doing nested locking.

yes, this is i (that "rbmm"), #opentowork

Re: Bug in reader/writer locks in Windows API

#107

Post by ajross »

ww520 wrote: post #63

The program has a bug. It's mixing atomic and non-atomic variables in the yield() checking loop. Non-atomic variables have no guarantee on cache consistency for different threads. This can cause the loop to run forever. struct ThreadTestData { int32_t numThreads = 0; std::shared_mutex sharedMutex = {}; std::atomic readCounter = 0; }; // child thread DoStuff() { data->readCounter.fetch_add(1); while (data->readCounter…

> Non-atomic variables have no guarantee on cache consistency for different threads

Atomicity and cache coherence are different things. "Atomic" means that the access to the element will be done in a single access and can only show exactly the state resulting from any other atomic access to the same value. For C syntax variables, this pretty much is limited to multi-word access (you also sometimes talk about atomic compare-and-set instructions, but those don't appear as part of the language per se).

Cache coherence is actually guaranteed on almost all systems, you don't need to worry about it on anything big enough to be running Windows (in the embedded world we get to fight it though).

The other demon in this space is memory reordering, but atomics don't speak to that at all.

Re: Bug in reader/writer locks in Windows API

#108

Post by ajross »

cesarb wrote: post #77
ww520 wrote: post #63

The program has a bug. It's mixing atomic and non-atomic variables in the yield() checking loop. Non-atomic variables have no guarantee on cache consistency for different threads. This can cause the loop to run forever. struct ThreadTestData { int32_t numThreads = 0; std::shared_mutex sharedMutex = {}; std::atomic readCounter = 0; }; // child thread DoStuff() { data->readCounter.fetch_add(1); while (data->readCounter…

> there's no memory barrier instruction to force its new value (5) to propagate to all CPU's running the threads. The equivalent of the memory barrier instructions is there, but it's hidden within the operating system code which creates and initializes a new thread. That is, the operating system ensures that the value in the current CPU (in this case, 5) is propagated to the CPU running the newly started thread, befo…

More generally, the OS is going to be doing some level of synchronization on its own, likely a spinlock, during the thread creation process. Those always include memory barriers, because otherwise the locks they define don't actually work on OO systems.

Re: Bug in reader/writer locks in Windows API

#109

Post by Const-me »

tialaramex wrote: post #39
AndrewStephens wrote: post #22

Subtle bugs in Reader/Writer locks do not surprise me. I worked on an in-house implementation based on Win32 (before C++11 and std::shared_mutex) and my recollection is that although the implementation sounds simple it is exceedingly easy to make subtle mistakes. The experience left me with such a bad feeling for shared locks that I tend to avoid them unless absolutely required. When I last tested std::shared_mutex,…

SRWLock seems like a situation where it's small enough for you to construct proofs that what you did is correct, and important enough that the enormous expense of such proof is justified. The C++ std::mutex provided in Microsoft's STL is ludicrously big, whereas SRWLocks are the same size as a pointer. In one sense being ludicrously big is great - less risk the std::mutex will mistakenly share a cache line with your…

C++ std::mutex provided in Microsoft's STL uses SRWLocks underneath, same size as a pointer.

Sadly, SRWLocks have yet another bug, they are very unfair. Write a loop which locks std::mutex inside the body, and no other thread will be able to grab the mutex despite the loop repeatedly releases then re-acquires the mutex. Critical sections are way more fair.

Re: Bug in reader/writer locks in Windows API

#110

Post by bongodongobob »

amluto wrote: post #23
rwmj wrote: post #17

Earlier quoted context omitted.

If it's any consolation, our company pays for 20,000 GMail licenses and we seem unable to escalate any issue at all to Google, even major issues such as their clearly not working spam filtering, emails being lost, incorrect deduplication of emails, or their random IMAP throttling.

Is there any vendor of actually good email-as-a-service? O365/Exchange/Outlook/Hotmail is a mess. Google has a support problem. Fastmail has an offline email problem. iCloud is not obviously suitable for professional use. What’s left?

How is O365/Exchange/Outlook a mess? As a user? As an admin? Genuinely curious. As an admin, I love O365.
Post reply on HN