Implement mechanism to wait on any of several futexes • Hacker News Board

Re: Implement mechanism to wait on any of several futexes

#151

Post by cesarb » Thu, Aug 01, 2019, 11:38 AM UTC

cryptonector wrote: post #131

monocasa wrote: post #121

Earlier quoted context omitted.

The vast majority of the interesting types _are_ files under Linux. You can use epoll with files, signals, timers, sockets, pipes, kernel semaphores, other epoll descriptors, page fault handling descriptors, etc. FDs are pretty much HANDLEs of the nix world.

HANDLEs are better. And on Windows you get HANDLEs to processes. We could have that in *nix land, you know.. something like opening /proc/$victim/status and using it to wait for the process and keep its PID from being reused until this is closed.

> We could have that in *nix land, you know.. something like opening /proc/$victim/status and using it to wait for the process and keep its PID from being reused until this is closed.

We are going to soon: "The 5.3 kernel also adds the ability to pass a pidfd to poll(), which will provide a notification when the process represented by that pidfd exits." (https://lwn.net/SubscriberLink/794707/905eb6b5b7287e77/)

Re: Implement mechanism to wait on any of several futexes

#152

Post by atombender » Thu, Aug 01, 2019, 12:58 PM UTC

monocasa wrote: post #121

jdsully wrote: post #117

Earlier quoted context omitted.

epoll only works with file descriptors and is therefore not equivalent. The whole point is WaitForMultipleObjects works with many different types.

The vast majority of the interesting types _are_ files under Linux. You can use epoll with files, signals, timers, sockets, pipes, kernel semaphores, other epoll descriptors, page fault handling descriptors, etc. FDs are pretty much HANDLEs of the nix world.

Last I heard, epoll() was, ironically, not useful with actual file descriptors, because file I/O isn't blocking on Linux [1]. I'm not sure if this problem has been remedied in recent years.

[1] https://stackoverflow.com/questions/8057892/epoll-on-regular...

Re: Implement mechanism to wait on any of several futexes

#153

Post by adrianmonk » Thu, Aug 01, 2019, 2:27 PM UTC

mjevans wrote: post #83

adrianmonk wrote: post #67

Earlier quoted context omitted.

Good point. I guess the emulator could request the profile, but then you either need a one-size-fits-all approach or somebody has to maintain a database of which games to enable it for.

A critical distinction is that it's not an emulator. "Compatibility layer" is a great description of what's actually happening. It's a (set of) native library(ies) which provide the ABI and call targets that are expected by the software and wrap around native OS functions to provide those. To use a metaphor, it's like someone made a replacement edge for a puzzle, which interlocks with a subset of existing pieces for…

Right. I've used the wrong terminology technically because the machine code is running natively. (I'm not aware of a succinct term for what WINE does, though, so I guess I'm being sloppy for convenience.)

Re: Implement mechanism to wait on any of several futexes

#154

Post by monocasa » Thu, Aug 01, 2019, 2:44 PM UTC

ithkuil wrote: post #144

dingu wrote: post #122

Earlier quoted context omitted.

what are you on about? I just gave you the specific examples: timerfd, eventfd, signalfd, inotify... these are all epollable fds in Linux? Futex is a special case because futexes themselves are quite special. There is no userspace equivalent to them in Windows anyway as has been mentioned. Windows Events are similar to what is provided by eventfd, but not as featureful.

Probably the core of the issue here is that VMS and derivatives tried to hard to fit everything into a generic handle/fd interface while UNIX historically came with a smaller core with many important API surfaces (like signals or timers) only relatively recently getting absorbed under the unified handle/fd interface, giving the impression it's an afterthought. It's also not something that works on all Unices (e.g. af…

Macos timers can be waited on via kqueue FWIW.

Re: Implement mechanism to wait on any of several futexes

#155

Post by badamp » Thu, Aug 01, 2019, 4:27 PM UTC

solipsism wrote: post #128

badamp wrote: post #98

Earlier quoted context omitted.

If you read the full thread that is a bit of a red herring and beside the point (thats why I said the conveyance of the performance implication was poor)... indeed window WFMO only supports 64 objects per call. They mention that the fd issue is due to leaking objects in many windows programs..which was an odd mention and a little off the main subject. The main motivator is performance. If eventfds performed better it…

Curious, how would a cache fix the fd leak issue?

Perhaps a better term would be “pool”. Anyway, what’s being leaked is “handles” or events not actually fds. You only actually need as many fds as the maximum possible number passed to a syscall. The mapping of handles/event objects in user space does not have to be 1:1 with the kernel resource.

Re: Implement mechanism to wait on any of several futexes

#156

Post by ithkuil » Thu, Aug 01, 2019, 4:33 PM UTC

monocasa wrote: post #154

ithkuil wrote: post #144

Earlier quoted context omitted.

Probably the core of the issue here is that VMS and derivatives tried to hard to fit everything into a generic handle/fd interface while UNIX historically came with a smaller core with many important API surfaces (like signals or timers) only relatively recently getting absorbed under the unified handle/fd interface, giving the impression it's an afterthought. It's also not something that works on all Unices (e.g. af…

Macos timers can be waited on via kqueue FWIW.

I know; but they cannot be closed with close, or passed via unix domain sockets to other processes etc etc. I _think_ this whole thread is about "purity", whatever that means.

Re: Implement mechanism to wait on any of several futexes

#157

Post by wiml » Thu, Aug 01, 2019, 4:54 PM UTC

kleer001 wrote: post #11

sjwright wrote: post #10

Earlier quoted context omitted.

The thing to remember is that version numbers aren’t decimals. They’re two or three integers separated by periods. 11 is higher than 2.

Harumph. Then it should be padded with zeros. So, 4.002 and 4.011 /grumpy edit: nope, it's fine as is. my experience in frame numbers, dates, and so forth do not scale to releases

Dates are the same, though — 01.15.2018 is not a real number, it's a sequence of integers. (And in many places they're not even written in most-to-least or least-to-most significant order! …I guess that makes it a tuple, not a sequence.)

Re: Implement mechanism to wait on any of several futexes

#158

Post by ComputerGuru » Sun, Aug 04, 2019, 12:34 AM UTC

monocasa wrote: post #138

ComputerGuru wrote: post #127

Earlier quoted context omitted.

Maybe, but only if you are doing it wrong. Pro-tip: look up hashed wheel timers.

Pro tip: implementing a hashed wheel timer yourself in user space comes with weird jitter because you can't atomically grab the system time and sleep for the next tick. Which is why all the sane OSes implement it or something like it in kernel space, including NT.

It’s not about implementing it more efficiently in a universal sense, it’s about making fewer unnecessary trips to the kernel and back. Context switches probably introduce more jitter than your userspace hashed timer wheel.