Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bug in reader/writer locks in Windows API

old.reddit.com

Post reply on HN
Print view View original

121–130 of 142 posts

Re: Bug in reader/writer locks in Windows API

#121

Post by NeillC »

Const-me wrote: post #109
tialaramex wrote: post #39

Earlier quoted context omitted.

SRWLock seems like a situation where it's small enough for you to construct proofs that what you did is correct, and important enough that the enormous expense of such proof is justified. The C++ std::mutex provided in Microsoft's STL is ludicrously big, whereas SRWLocks are the same size as a pointer. In one sense being ludicrously big is great - less risk the std::mutex will mistakenly share a cache line with your…

C++ std::mutex provided in Microsoft's STL uses SRWLocks underneath, same size as a pointer. Sadly, SRWLocks have yet another bug, they are very unfair. Write a loop which locks std::mutex inside the body, and no other thread will be able to grab the mutex despite the loop repeatedly releases then re-acquires the mutex. Critical sections are way more fair.

Critical sections are unfair also. Could be differences in spin count you are seeing.

Re: Bug in reader/writer locks in Windows API

#122

Post by charcircuit »

Tringi wrote: post #112
charcircuit wrote: post #36

Earlier quoted context omitted.

Microsoft engineers have emails too.

And Twitter/X accounts. I have had a lot more things fixed in Windows or MSVC from nagging devs on there than from reporting through any official channel.

+1 if you want to reach a real engineer they are going to be spending their free time on sites like X and not on sites like some community feedback and bug reporting form

Re: Bug in reader/writer locks in Windows API

#123

Post by Iwan-Zotow »

userbinator wrote: post #64
nightowl_games wrote: post #55

Earlier quoted context omitted.

How did you patch the kernel? Like how is that possible?

With a hex editor, debugger, and skills that most developers these days seem to lack. I patched the kernel in memory first, using a kernel debugger, to verify my fix worked before editing the file on disk.

Aren't kernel modules signed?

Re: Bug in reader/writer locks in Windows API

#124

Post by Tringi »

NeillC wrote: post #120
quotemstr wrote: post #118

Earlier quoted context omitted.

> My overall rule of thumb is that in the presence of any exclusive acquires you can never assume a shared acquire is compatible with any other shared acquire. Contract #1: In reader writer locks in general, after I share-acquire a lock, I know that there are no active exclusive owners of that lock and won't be until I release my shared lock. I can also expect that as long as I hold this shared lock, other threads ca…

It's typical not to allow a second share acquire to proceed if we have an exclusive waiter: t1: share t2: exclusive so waits t3: share also waits Thats how we arrive at the rule that shared acquires in anything, but a trivial system (no exclusive acquires) may not be compatible. So, contract #1 is typically not satisfied. Of course, this particular case is slightly different and so you might decide to support it.

Blocking new readers when a writer arrives is perfectly good and desirable. Blocking readers when the writer finishes, and there isn't any new queued, is definitely not.

Re: Bug in reader/writer locks in Windows API

#125

Post by Tringi »

NeillC wrote: post #121
Const-me wrote: post #109

Earlier quoted context omitted.

C++ std::mutex provided in Microsoft's STL uses SRWLocks underneath, same size as a pointer. Sadly, SRWLocks have yet another bug, they are very unfair. Write a loop which locks std::mutex inside the body, and no other thread will be able to grab the mutex despite the loop repeatedly releases then re-acquires the mutex. Critical sections are way more fair.

Critical sections are unfair also. Could be differences in spin count you are seeing.

Some time ago I did some test, 256 threads competing on a small number of cache lines, and found out that all, CreateMutex, CRITICAL_SECTION and SRWLOCK, were quite fair.

The most successful thread was only 25%, 15% and 9% ahead of the least successful one. On the contrary, in my simple usermode spinlock the unfairness would be 1000% or even 2000%.

Re: Bug in reader/writer locks in Windows API

#126

Post by runaum »

Iwan-Zotow wrote: post #123
userbinator wrote: post #64

Earlier quoted context omitted.

With a hex editor, debugger, and skills that most developers these days seem to lack. I patched the kernel in memory first, using a kernel debugger, to verify my fix worked before editing the file on disk.

Aren't kernel modules signed?

not in xp, iirc, vista was the first release requiring it

Re: Bug in reader/writer locks in Windows API

#127

Post by AnthonyMouse »

deckar01 wrote: post #116
AnthonyMouse wrote: post #45

Earlier quoted context omitted.

The fundamental problem here is that they have a billion users and most of them don't know what they're talking about. If you create a simple way to contact the company it will soon be full of messages from end users who can't even articulate what their problem is but it's usually some kind of malware or user error and is definitely not a problem with whatever component they're reporting the issue against. What you r…

Not just high friction, but a formal verification like certification would be interesting.

Nah, you don't want bureaucratic gatekeeping. You want the 14 year old in Kenya or Ukraine with technical competence but no ID to be able to report the bug he finds.

The point is to exclude people who don't know what they're talking about, not people who can't pay registration fees or produce documents.

Re: Bug in reader/writer locks in Windows API

#128

Post by zozbot234 »

NeillC wrote: post #93
zozbot234 wrote: post #52

Earlier quoted context omitted.

Why does it have to use bit test and set and interleave with other threads, though. AIUI you can use a CAS loop to implement any RMW atomically over word-sized (or double word sized, on many platforms) data. That seems like a no-brainer. For comparison, the Rust implementation for lightweight RWLocks on futex-capable *nix platforms is here: https://doc.rust-lang.org/stable/src/std/sys/unix/locks/fute... It sets the "…

We use bit test and set as it causes us to get the cache line exclusive. This avoids using the prefetch write before fetching if we were to use CompareExchange. somebody was trying to talk to me about this stuff this morning but I didn't understand him. You can't expect a reader to always be compatible with other readers otherwise you livelock with a constant stream of readers. So we become incompatible. I am unsure…

Interesting comment for sure. Of course if this is intended behaviour it should at least be properly documented, since other implementations don't seem to do this random "upgrading" of a shared to an exclusive lock and it does create an issue whenever readers might be waiting on one another while holding the lock as shown in OP's code.

Re: Bug in reader/writer locks in Windows API

#129

Post by holigot »

jeff_tyrrill wrote: post #119
bongodongobob wrote: post #110

Earlier quoted context omitted.

How is O365/Exchange/Outlook a mess? As a user? As an admin? Genuinely curious. As an admin, I love O365.

Outlook is a once-great product that has been left to rot by a Microsoft with little lineage to the great company from the 90s and early 00s that created it. Outlook debuted Cached Exchange Mode in the early 00s, popularizing "offline first" before it was known as that. Now: The "new" Outlook can't even show folder unread counts correctly, even when fully online. It seems to only load a small subset of messages local…

Especially the point about Fastmail really is a good point! You nailed it with your comment. With JMAP in particular, they have been given huge potential with a huge feature set but have not implemented it (especially not in their own products). Especially Offline Usage...

Re: Bug in reader/writer locks in Windows API

#130

Post by holigot »

amluto wrote: post #23
rwmj wrote: post #17

Earlier quoted context omitted.

If it's any consolation, our company pays for 20,000 GMail licenses and we seem unable to escalate any issue at all to Google, even major issues such as their clearly not working spam filtering, emails being lost, incorrect deduplication of emails, or their random IMAP throttling.

Is there any vendor of actually good email-as-a-service? O365/Exchange/Outlook/Hotmail is a mess. Google has a support problem. Fastmail has an offline email problem. iCloud is not obviously suitable for professional use. What’s left?

Good question.

It really is a tough choice. Microsoft does have the best products for their and mobile platforms (including iOS,iPadOS) with really good Offline Features and so on. But it still is Microsoft and Exchange behind...

A better product is Fastmail. But also US Servers and AUS company and a lot of downsides concerning offline usage and other privacy points.

What else? I try to avoid Google as much as I can.... Well there is Proton with it's Proton Suite getting better and better. There is german mailbox.org (worse 2FA) but overall a good privacy mail provider with own domains. There is migadu.com from Switzerland with EU Servers (rented at OVH Data Centers) and some other players like Tuta and others...

Post reply on HN