Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Implement mechanism to wait on any of several futexes

lkml.org

Post reply on HN
Print view View original

131–140 of 158 posts

Re: Implement mechanism to wait on any of several futexes

#131

Post by cryptonector »

monocasa wrote: post #121
jdsully wrote: post #117

Earlier quoted context omitted.

epoll only works with file descriptors and is therefore not equivalent. The whole point is WaitForMultipleObjects works with many different types.

The vast majority of the interesting types _are_ files under Linux. You can use epoll with files, signals, timers, sockets, pipes, kernel semaphores, other epoll descriptors, page fault handling descriptors, etc. FDs are pretty much HANDLEs of the nix world.

HANDLEs are better. And on Windows you get HANDLEs to processes. We could have that in *nix land, you know.. something like opening /proc/$victim/status and using it to wait for the process and keep its PID from being reused until this is closed.

Re: Implement mechanism to wait on any of several futexes

#132

Post by jdsully »

monocasa wrote: post #123
jdsully wrote: post #120

Earlier quoted context omitted.

While the Unix philosophy is that everything is a file - that is not the case with Linux. Futexs being a good example. There was an attempt to integrate them but it was done incorrectly with inherent race conditions and abandoned. The VMS and NT philosophies of everything being an object are a bit more general and easier to follow in practice.

Futexes currently have _no_ kernel state besides in the threads that are currently waiting on them. There's no futex_create system call for instance. It's litreally just a call to "sleep until this memory address changes or a timeout occurs". There's not really anything to make a type around, which is why FUTEX_FD was doomed to failure and they rightly backed it out.

You may be surprised to learn that some windows handles are actually just addresses with no kernel state too. A great example is an HMODULE.

In fact windows actually reserves a block of user address space that will never be allocated to disambiguate memory vs non memory handles.

Its really a pitty these two great systems refuse to learn from each other.

Re: Implement mechanism to wait on any of several futexes

#133

Post by dingu »

jdsully wrote: post #132
monocasa wrote: post #123

Earlier quoted context omitted.

Futexes currently have _no_ kernel state besides in the threads that are currently waiting on them. There's no futex_create system call for instance. It's litreally just a call to "sleep until this memory address changes or a timeout occurs". There's not really anything to make a type around, which is why FUTEX_FD was doomed to failure and they rightly backed it out.

You may be surprised to learn that some windows handles are actually just addresses with no kernel state too. A great example is an HMODULE. In fact windows actually reserves a block of user address space that will never be allocated to disambiguate memory vs non memory handles. Its really a pitty these two great systems refuse to learn from each other.

What does that have to do with anything? You wouldn't use WFMO to implement a futex in Windows. And regardless of anything in Linux, defending WFMO seems like a strange hill to die on. It's not a great API.

I'm calling your bluff too. Which one of the supported waitable handles in Windows are just addresses with no other state? I'm more surprised because they all need an access mask at a minimum, and I thought they all involve an ObCreateObject, even a Mutant... these dusty corners of the kernel are visible through the DDK. Anyway, I would be glad to be shown wrong.

Re: Implement mechanism to wait on any of several futexes

#134

Post by monocasa »

jdsully wrote: post #132
monocasa wrote: post #123

Earlier quoted context omitted.

Futexes currently have _no_ kernel state besides in the threads that are currently waiting on them. There's no futex_create system call for instance. It's litreally just a call to "sleep until this memory address changes or a timeout occurs". There's not really anything to make a type around, which is why FUTEX_FD was doomed to failure and they rightly backed it out.

You may be surprised to learn that some windows handles are actually just addresses with no kernel state too. A great example is an HMODULE. In fact windows actually reserves a block of user address space that will never be allocated to disambiguate memory vs non memory handles. Its really a pitty these two great systems refuse to learn from each other.

The handle is a _kernel_ address... backed by a validly constructed kernel object.

EDIT: And since you dirty edited, can you point to an example of HMODULE or any other dataless HANDLE being used on the syscall interface?

Re: Implement mechanism to wait on any of several futexes

#135

Post by dingu »

jdsully wrote: post #132
monocasa wrote: post #123

Earlier quoted context omitted.

Futexes currently have _no_ kernel state besides in the threads that are currently waiting on them. There's no futex_create system call for instance. It's litreally just a call to "sleep until this memory address changes or a timeout occurs". There's not really anything to make a type around, which is why FUTEX_FD was doomed to failure and they rightly backed it out.

You may be surprised to learn that some windows handles are actually just addresses with no kernel state too. A great example is an HMODULE. In fact windows actually reserves a block of user address space that will never be allocated to disambiguate memory vs non memory handles. Its really a pitty these two great systems refuse to learn from each other.

Ah you edited your response? An HMODULE.. That is weak dude. Calling and HMODULE a HANDLE is cheating... by that definition a UINT is a HANDLE since you can cast it.

Can you use HMODULEs orthogonally to common APIs that use Handles? That's what really matters. Just because something is 4 or 8 bytes wide and you call it the same thing isn't interesting. Like can you pass HMODULE to WaitForSingleObject? Oh Ok, I guess that's mean because to be fair, what does "waiting" on a DLL mean.

Ok, well surely you can pass that HMODULE to CloseHandle, at least closing should be orthogonal.. Why don't you try that? I'll wait.

So what point are you trying to make here? It sound like you're just jerking everyone around, tbh.

Edit: Moreover, it's a bit ridiculous to say that an HMODULE is just an address with no kernel state. It uniquely identifies the loaded DLL, so it a key to a tremendous amount of kernel bookkeeping about the loaded module.

Re: Implement mechanism to wait on any of several futexes

#136

Post by monocasa »

mehrdadn wrote: post #111
monocasa wrote: post #109

Earlier quoted context omitted.

It's my way of trying to keep the conversation on the topic of the actual article, rather than turning it into a generic Linus v Cutler boxing match where both of their syscall interfaces are on the table. And staying on the topic of mutexes, that "just spawn another thread for every 64 items you want to wait on" (which I already knew about) is about the hackiest shit ever. (And FWIW, I started off as a Win32 develop…

> And staying on the topic of mutexes, that "just spawn another thread for every 64 items you want to wait on" (which I already knew about) is about the hackiest shit ever. Is this surprising to you? I already told you it would be a hack because I told you there is a better and proper solution for the actual problem you were encountering. You're stubbornly insisting for no reason on actively doing something bizarre ,…

Windows runs on single socket systems with 128 cores. If you can't think of some time you might need to wait on more than 64 items at a time, you only have your own lack of imagination to blame.

Re: Implement mechanism to wait on any of several futexes

#137

Post by lostmsu »

dpcx wrote: post #95
lostmsu wrote: post #78

Earlier quoted context omitted.

I think this is very backwards. When solving major problems details of taste are usually a blocker or waste of time.

I think a law that says speeding is very backwards. When I'm late, details like how far over the speed limit I'm going are a blocker or a waste of time. The kernel team came up with rules about how code is formatted. If you don't follow the rules, they are under no obligation to allow your code to be merged in to the main repo, and in fact, are within their rights to reject it. I take the initial response more as a "…

Unlike code style, speed limits are not a matter of taste.

Re: Implement mechanism to wait on any of several futexes

#138

Post by monocasa »

ComputerGuru wrote: post #127
monocasa wrote: post #97

Earlier quoted context omitted.

Off the top of my head, timers?

Maybe, but only if you are doing it wrong. Pro-tip: look up hashed wheel timers.

Pro tip: implementing a hashed wheel timer yourself in user space comes with weird jitter because you can't atomically grab the system time and sleep for the next tick. Which is why all the sane OSes implement it or something like it in kernel space, including NT.

Re: Implement mechanism to wait on any of several futexes

#139

Post by mehrdadn »

monocasa wrote: post #136
mehrdadn wrote: post #111

Earlier quoted context omitted.

> And staying on the topic of mutexes, that "just spawn another thread for every 64 items you want to wait on" (which I already knew about) is about the hackiest shit ever. Is this surprising to you? I already told you it would be a hack because I told you there is a better and proper solution for the actual problem you were encountering. You're stubbornly insisting for no reason on actively doing something bizarre ,…

Windows runs on single socket systems with 128 cores. If you can't think of some time you might need to wait on more than 64 items at a time, you only have your own lack of imagination to blame.

The number of mutexes I generally wait on has nothing to do with how many cores my computer has. But I'm giving up on the hope that you'll ever share with us your wonderful imagination.

Re: Implement mechanism to wait on any of several futexes

#140

Post by jdsully »

dingu wrote: post #133
jdsully wrote: post #132

Earlier quoted context omitted.

You may be surprised to learn that some windows handles are actually just addresses with no kernel state too. A great example is an HMODULE. In fact windows actually reserves a block of user address space that will never be allocated to disambiguate memory vs non memory handles. Its really a pitty these two great systems refuse to learn from each other.

What does that have to do with anything? You wouldn't use WFMO to implement a futex in Windows. And regardless of anything in Linux, defending WFMO seems like a strange hill to die on. It's not a great API. I'm calling your bluff too. Which one of the supported waitable handles in Windows are just addresses with no other state? I'm more surprised because they all need an access mask at a minimum, and I thought they a…

I’m not sure why theres such animosity in your replies. The goal here is not to defend windows but to point out that handles need not aways refer to kernel objects.

It would be possible to provide an orthogonal handle based API even if windows doesn’t always live up to that ideal.

Ultimately it seems these systems are just too big to maintain a cohesive design - but it is still an ideal to aspire to.

Post reply on HN