Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Implement mechanism to wait on any of several futexes

lkml.org

Post reply on HN
Print view View original

121–130 of 158 posts

Re: Implement mechanism to wait on any of several futexes

#121

Post by monocasa »

jdsully wrote: post #117
jlokier wrote: post #112

Earlier quoted context omitted.

Linux has WaitForMultipleObjects already: fds are equivalent to Windows HANDLEs, and WaitForMultipleObjects() is equivalent to poll(). But, Linux also has epoll() which scales better for non-trivial numbers of things, and Windows has IOCP. So WaitForMultipleObjects isn't particularly special. Both Windows and Linux have things that don't work with these interfaces. Nonetheless, Linux has been trending towards "waitin…

epoll only works with file descriptors and is therefore not equivalent. The whole point is WaitForMultipleObjects works with many different types.

The vast majority of the interesting types _are_ files under Linux. You can use epoll with files, signals, timers, sockets, pipes, kernel semaphores, other epoll descriptors, page fault handling descriptors, etc.

FDs are pretty much HANDLEs of the nix world.

Re: Implement mechanism to wait on any of several futexes

#122

Post by dingu »

jdsully wrote: post #120
dingu wrote: post #119

Earlier quoted context omitted.

You are confused. File descriptors on linux represent "many different types." They are not just for "disk files". Please see signalfd, timerfd, eventfd, inotify, let alone sockets (which themselves represent things other than IP sockets). FD is essentially like handle. epoll therefore works with many different types.

While the Unix philosophy is that everything is a file - that is not the case with Linux. Futexs being a good example. There was an attempt to integrate them but it was done incorrectly with inherent race conditions and abandoned. The VMS and NT philosophies of everything being an object are a bit more general and easier to follow in practice.

what are you on about? I just gave you the specific examples: timerfd, eventfd, signalfd, inotify... these are all epollable fds in Linux?

Futex is a special case because futexes themselves are quite special. There is no userspace equivalent to them in Windows anyway as has been mentioned. Windows Events are similar to what is provided by eventfd, but not as featureful.

Re: Implement mechanism to wait on any of several futexes

#123

Post by monocasa »

jdsully wrote: post #120
dingu wrote: post #119

Earlier quoted context omitted.

You are confused. File descriptors on linux represent "many different types." They are not just for "disk files". Please see signalfd, timerfd, eventfd, inotify, let alone sockets (which themselves represent things other than IP sockets). FD is essentially like handle. epoll therefore works with many different types.

While the Unix philosophy is that everything is a file - that is not the case with Linux. Futexs being a good example. There was an attempt to integrate them but it was done incorrectly with inherent race conditions and abandoned. The VMS and NT philosophies of everything being an object are a bit more general and easier to follow in practice.

Futexes currently have _no_ kernel state besides in the threads that are currently waiting on them. There's no futex_create system call for instance. It's litreally just a call to "sleep until this memory address changes or a timeout occurs". There's not really anything to make a type around, which is why FUTEX_FD was doomed to failure and they rightly backed it out.

Re: Implement mechanism to wait on any of several futexes

#124

Post by caf »

jdsully wrote: post #120
dingu wrote: post #119

Earlier quoted context omitted.

You are confused. File descriptors on linux represent "many different types." They are not just for "disk files". Please see signalfd, timerfd, eventfd, inotify, let alone sockets (which themselves represent things other than IP sockets). FD is essentially like handle. epoll therefore works with many different types.

While the Unix philosophy is that everything is a file - that is not the case with Linux. Futexs being a good example. There was an attempt to integrate them but it was done incorrectly with inherent race conditions and abandoned. The VMS and NT philosophies of everything being an object are a bit more general and easier to follow in practice.

Windows has WaitOnAddress() which is similar to a futex, and likewise isn't a HANDLE.

"File descriptor" is just a weird spelling of "handle", for historical reasons.

Re: Implement mechanism to wait on any of several futexes

#125

Post by jdsully »

wbl wrote: post #105
jdsully wrote: post #89

Earlier quoted context omitted.

Thats my point. They should just bite the bullet and implement WaitForMultipleObjects instead of having all these disjoint APIs. Plus sometimes you want to wait for both and this is very difficult with Linux. Its also telling Linux has gone through the whole select/poll/epoll madness while WaitForMultipleObjects has worked well in windows NT since the 90s. Its a proven design.

Poll and select predate linux. The issues with epoll were in the 2000's and kqueue has been very stable by comparison.

Kqueue is quite nice. BSD is much better designed and I enjoy working with it more. Linux suffers from people doing just enough to solve their specific problem and then no more. The result is APIs are not orthogonal and follow differing styles. Sometimes they don’t even work at all (e.g. futex fds)

I admire what Linux accomplishes functionally. But it is just as ugly as win32 in my opinion. The two are really more alike in their pragmatism than they would care to admit - but mutual hatred prevents learning from each others mistakes.

Re: Implement mechanism to wait on any of several futexes

#127

Post by ComputerGuru »

monocasa wrote: post #97
mehrdadn wrote: post #96

Earlier quoted context omitted.

Right, but when do you need to simultaneously wait on more than 64 things other than for I/O? I've never had to wait on that many mutexes... (P.S. there is a really ugly way to get around this on Windows if for some bizarre reason you really need to, which is to have 1 thread per 64 handles, then wait on the thread handle instead. I've never found a need do even get close to doing such a thing though.)

Off the top of my head, timers?

Maybe, but only if you are doing it wrong. Pro-tip: look up hashed wheel timers.

Re: Implement mechanism to wait on any of several futexes

#128

Post by solipsism »

badamp wrote: post #98
Taniwha wrote: post #81

Earlier quoted context omitted.

They point out that they already have an implementation that does just this .... and it fails on some programs due to running out of file descriptors (they have one program that needs ~1 million of them ...)

If you read the full thread that is a bit of a red herring and beside the point (thats why I said the conveyance of the performance implication was poor)... indeed window WFMO only supports 64 objects per call. They mention that the fd issue is due to leaking objects in many windows programs..which was an odd mention and a little off the main subject. The main motivator is performance. If eventfds performed better it…

Curious, how would a cache fix the fd leak issue?

Re: Implement mechanism to wait on any of several futexes

#129

Post by cryptonector »

I'd like a way to wait on multiple condition variables. Windows has this... Or to treat condition variables as file descriptors so that select/poll/epoll can be used to wait on them (you'd get a notification of a signal, though it might be spurious, so you still have to take the lock and check the condition before you consider the CV signaled).

Re: Implement mechanism to wait on any of several futexes

#130

Post by wbl »

jdsully wrote: post #125
wbl wrote: post #105

Earlier quoted context omitted.

Poll and select predate linux. The issues with epoll were in the 2000's and kqueue has been very stable by comparison.

Kqueue is quite nice. BSD is much better designed and I enjoy working with it more. Linux suffers from people doing just enough to solve their specific problem and then no more. The result is APIs are not orthogonal and follow differing styles. Sometimes they don’t even work at all (e.g. futex fds) I admire what Linux accomplishes functionally. But it is just as ugly as win32 in my opinion. The two are really more al…

I've never worked with Win32, just read about the more core bits of NT, and those seem quite nice although perhaps overgeneral. But I've always had a soft spot for oddball systems.
Post reply on HN