Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Implement mechanism to wait on any of several futexes

lkml.org

Post reply on HN
Print view View original

81–90 of 158 posts

Re: Implement mechanism to wait on any of several futexes

#81

Post by Taniwha »

badamp wrote: post #66
Asooka wrote: post #62

I know the patch mentions interactive multimedia applications (games) in particular, but an actual mechanism to implement WaitForMultipleObjects on Linux would be very welcome for many high-performance multi-threaded applications. Say you have one worker thread per CPU core. On Windows, each thread would get an Event object and you would WaitOnMultiple to be able to act on the first unit of work that was complete. On…

Linux already has what you’re talking about with eventfd and epoll. In Linux each thread can get an eventfd and you can POLLIN all of them. In fact I would argue that using futexes is the “roll your own solution” using lower level primitives (and easier to fuckup) much more so than eventfd and epoll. As mentioned somewhat poorly in the post, using futexes gives a performance boost which is not surprising since they a…

They point out that they already have an implementation that does just this .... and it fails on some programs due to running out of file descriptors (they have one program that needs ~1 million of them ...)

Re: Implement mechanism to wait on any of several futexes

#82

Post by mirashii »

kbenson wrote: post #73
rbanffy wrote: post #23

Earlier quoted context omitted.

I got that. I intended to go for "actually useful".

Source 2 has a linux port I believe, and it's an engine that other games could (do?) use to get easy linux support for a lot of their codebase. I think that counts for actually useful. Depending on what they mean by "massively-threaded", that might cover some other popular network applications that thread to handle requests, but I'm not sure how much work they would put into a linux specific solution if it complicate…

> (do?)

Indeed. See Dota 2.

Re: Implement mechanism to wait on any of several futexes

#83

Post by mjevans »

adrianmonk wrote: post #67
TrueDuality wrote: post #65

Earlier quoted context omitted.

Ideally yes, but remember that a lot of these games aren't designed for Linux but are effectively running in a compatibility layer for Windows. There is no way for the programmers to know that they need to be interacting with a Linux window manager's behavior.

Good point. I guess the emulator could request the profile, but then you either need a one-size-fits-all approach or somebody has to maintain a database of which games to enable it for.

A critical distinction is that it's not an emulator. "Compatibility layer" is a great description of what's actually happening. It's a (set of) native library(ies) which provide the ABI and call targets that are expected by the software and wrap around native OS functions to provide those.

To use a metaphor, it's like someone made a replacement edge for a puzzle, which interlocks with a subset of existing pieces for another puzzle rather than someone making a table-sandbox within which to use the the entire initial puzzle.

Re: Implement mechanism to wait on any of several futexes

#84

Post by zamalek »

swiley wrote: post #29

This looks to me like the main change is to make it easier to create mutexes with timeouts. Isn’t a mutex timing out an indication that: a) a lock wasn’t needed in the first place or b) the program is incorrect? It feels more like they just want the api to match win32 better but most of the multithreaded programming I’ve done lately has just used go’s channels so I totally could be missing something.

One valid scenario for mutexes timing out is stealing:

1. Each CPU first attempts to take an allocated object from its own pool, using a 1ms timeout to acquire the mutex on the pool.

2. If that fails, the CPU attempts to steal from the other CPU pools, using a zero timeout (immediately return if the mutex can't be taken).

3. If that fails, acquire the mutex on its own pool with an infinite timeout.

This is definitely correct and is much faster than any of the lock-free approaches I could come up with. I guess the general class of problems where mutex timeouts help is when you have "many valid options, some of which are being used by others."

Re: Implement mechanism to wait on any of several futexes

#85

Post by mjevans »

Taniwha wrote: post #81
badamp wrote: post #66

Earlier quoted context omitted.

Linux already has what you’re talking about with eventfd and epoll. In Linux each thread can get an eventfd and you can POLLIN all of them. In fact I would argue that using futexes is the “roll your own solution” using lower level primitives (and easier to fuckup) much more so than eventfd and epoll. As mentioned somewhat poorly in the post, using futexes gives a performance boost which is not surprising since they a…

They point out that they already have an implementation that does just this .... and it fails on some programs due to running out of file descriptors (they have one program that needs ~1 million of them ...)

I recall higher performance browsers also use up large numbers of FDs; I suspect it might be for this very reason.

Re: Implement mechanism to wait on any of several futexes

#86

Post by waterhouse »

MadSudaca wrote: post #50
tambre wrote: post #31

Earlier quoted context omitted.

> Wine emulator Wine stands for "Wine Is Not an Emulator".

The Wine is Not an Emulator emulator

Yeah, seriously. It's not emulating a processor, but it is emulating an API. I think the only reason it's not called Windows Emulator is (probably well-founded) concern about getting sued for trademark violation.

Citation with detailed history: https://news.ycombinator.com/item?id=13476390

Re: Implement mechanism to wait on any of several futexes

#87

Post by danappelxx »

Shorel wrote: post #71
gnufied wrote: post #46

Personally for me - the problematic part of gaming on Linux has been input(i.e mouse) latency and acceleration profile. I am not sure if this is just my experience but when using libinput on Fedora for example - the cursor movement is not exactly precise. This is not obvious when working but while gaming this is a deal breaker.

Games in Windows can get raw mouse input just by writing the code for it, and Linux games usually don't because that would require root permissions, to add the user to the input group, etc. It is a security issue and an X-Window design issue.

Oblivious question: is there any notion of granulated permissions, ie a ‘mouse’ user group that the game could add itself to? Seems like something like this shouldn’t be a deal-breaker.

Re: Implement mechanism to wait on any of several futexes

#88

Post by monocasa »

jdsully wrote: post #80

I like how they won’t say the windows API’s name. WaitForMultipleObjects is quite nice - an epoll that works with all sorts of things not just fds. This is kind of a half assed implementation - sometimes I wish Linux would admit Windows has a good idea once in a while.

This isn't really like WaitForMultipleObjects; WFMO is more like epoll with a whole lot more restrictions.

This legit isnt really like anything in Windows, the closest thing I can think of is you'd be able to do something like this with XOK wake predicates.

Re: Implement mechanism to wait on any of several futexes

#89

Post by jdsully »

monocasa wrote: post #88
jdsully wrote: post #80

I like how they won’t say the windows API’s name. WaitForMultipleObjects is quite nice - an epoll that works with all sorts of things not just fds. This is kind of a half assed implementation - sometimes I wish Linux would admit Windows has a good idea once in a while.

This isn't really like WaitForMultipleObjects; WFMO is more like epoll with a whole lot more restrictions. This legit isnt really like anything in Windows, the closest thing I can think of is you'd be able to do something like this with XOK wake predicates.

Thats my point. They should just bite the bullet and implement WaitForMultipleObjects instead of having all these disjoint APIs. Plus sometimes you want to wait for both and this is very difficult with Linux.

Its also telling Linux has gone through the whole select/poll/epoll madness while WaitForMultipleObjects has worked well in windows NT since the 90s. Its a proven design.

Re: Implement mechanism to wait on any of several futexes

#90

Post by monocasa »

jdsully wrote: post #89
monocasa wrote: post #88

Earlier quoted context omitted.

This isn't really like WaitForMultipleObjects; WFMO is more like epoll with a whole lot more restrictions. This legit isnt really like anything in Windows, the closest thing I can think of is you'd be able to do something like this with XOK wake predicates.

Thats my point. They should just bite the bullet and implement WaitForMultipleObjects instead of having all these disjoint APIs. Plus sometimes you want to wait for both and this is very difficult with Linux. Its also telling Linux has gone through the whole select/poll/epoll madness while WaitForMultipleObjects has worked well in windows NT since the 90s. Its a proven design.

I mean.. epoll is already better than WFMO. It doesn't have that silly 64 item cap, or O(N) entry and exit on the wait like it's 1995 all over again. (And that's not to say that epoll is the pinnacle of design, just that WFMO is worse. We can do better than both)

And this better too, you don't need to register futexes with the kernel, the futex wait call is just "I want to sleep until the data at this memory address changes", and now you can say "I want to sleep until _these addresses_ change". NT doesn't give you any equivalent, you need to build in kernel objects to do the same thing.

And if you want to wait for both, you just wrap it into an eventfd and epoll on that.

Post reply on HN