Fixing stutters in Papers Please on Linux • Hacker News Board

Re: Fixing stutters in Papers Please on Linux

#111

Post by spijdar » Sun, Jan 02, 2022, 8:45 PM UTC

facorreia wrote: post #100

This reminds me of the recent Linus Tech Tips series on gaming on Linux[1]. Their conclusion is that although many games work out of the box (although usually not at launch), Linux is not ready for mainstream gamers. Not many people would have the expertise or the interest to troubleshoot the problem as OP did. [1] https://www.youtube.com/watch?v=Rlg4K16ujFw

It definitely isn't. I've been a huge linux nerd since my preteens in the late 2000s, I jumped on to squeeze more performance out of the thoroughly mediocre hardware I had access to. I wanted to program, and I found Visual Studio to be incomprehensibly dense and confusing, while Linux tools were so much simpler, with GCC, GEdit, makefiles and the like being more to my liking. I fell deep into the rabbit hole, learned emacs, then vim (it was more responsive on my intel atom-powered netbook), became a "shell guru", eventually went to college at 16 and started doing cybersecurity work/pentesting professionally. I've even made a tiny contribution to the Linux kernel, which I'm pretty proud of.

All this anecdata to say, I consider myself pretty okay at using Linux, I "prefer" Linux, but I don't use Linux for gaming. Not unless it makes sense. I play Minecraft on Linux, and FOSS games that were developed on Linux. There's a POWER9 desktop on my desk that runs Linux, and all my professional and hobby work goes there. I love it.

But any commercial games? They go on my old college-days Intel desktop, running Win10. I can do the work to get games running on Linux, but why bother? Like Linus says in that video, when I have time to play video games, I really don't want to pull out a debugger and strace and crap to do more $DAYJOB work.

Not to say I never do that for fun. I do. I've done some work with https://github.com/ptitSeb/box86, and that involves a similar process. But I just frankly don't find doing it to your average Steam game to be very fun. Sometimes the muse strikes, usually it doesn't.

And for your average Linux user, much less your average computer user overall, you can forget about it. IMO, unless you have a strong ideological reason to only use FOSS OSes (and all the power to you!), the reason you use Linux is because it's a vastly superior tool for certain problems.

Playing your average commercial game is not one of them.

Re: Fixing stutters in Papers Please on Linux

#112

Post by ux » Sun, Jan 02, 2022, 8:49 PM UTC

Great post, learned a bunch of things. Only one blog post, and no RSS (yet?) unfortunately.

Re: Fixing stutters in Papers Please on Linux

#113

Post by bregma » Sun, Jan 02, 2022, 8:58 PM UTC

Sjonny wrote: post #61

nemetroid wrote: post #54

Earlier quoted context omitted.

SDL has three methods for detecting input devices [1]: udev, inotify, and, as a fallback, enumerating /dev/input. It seems like Papers, Please uses a statically linked version of SDL, without udev or inotify support compiled in. 1: https://github.com/libsdl-org/SDL/blob/d0de4c625ad26ef540166...

I've been wondering.. is it possible to write something to override the statically linked functions? In this case, most (if not all) functions have an SDL_ prefix. Would it be possible to LD_PRELOAD a library that loads a shared version of SDL and goes over all the function pointers to move them point them to a new location? Is there a tool for this?

Static linking means the features of the Linux dynamic loader, like using the environment variable LD_PRELOAD to pre-load a dynamic library, are not going to have any effect.

Re: Fixing stutters in Papers Please on Linux

#114

Post by Denvercoder9 » Sun, Jan 02, 2022, 9:06 PM UTC

londons_explore wrote: post #78

tankenmate wrote: post #19

Earlier quoted context omitted.

I just did a quick check the posted fix is not in the most recent -rc branch in the public git repo.

This is the issue with using mailing lists... Large numbers of perfectly good fixes, embodying many hours of effort, just get missed and forgotten about. At least with GitHub PR's, every request either needs to be merged or rejected.

The modern trend is autoclosing PR's after they haven't had any activity in X months, so there's not much difference with mailing lists anymore...

Re: Fixing stutters in Papers Please on Linux

#115

Post by loeg » Sun, Jan 02, 2022, 9:09 PM UTC

CyberRabbi wrote: post #90

loeg wrote: post #89

Earlier quoted context omitted.

> Blocking simply means that the specification does not guarantee an upper bound on the completion time. I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking. > I think this is an instance of confusing what should be with what is. Who is doing the confusing? I said "should be." Are you saying they're fast now but should be slow? Why? > The reality is…

> I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking. When it comes to expecting a specific duration, getpid() is blocking. If you run getpid() in a tight loop and then have performance issues you can’t reasonably blame the system. > This isn't a portable program; it's a Linux program But the interface is a portable interface > POSIX does not manda…

> Today the issue was close() just happened to be too “slow.” If the amount of input devices were higher, let’s say 2x more, then the same issue would have manifested even if close() were 2x “faster.” No matter how fast you make close() there is a situation in which this issue would manifest itself.

Close, on an fd for which no asynchronous IO has occurred, should be 10000x faster, or more. It’s unlikely a user will have even 100 real input devices. I agree the algorithm leaves something to be desired, but the only reason it is user-visible is the performance bug in Linux.

I’ve worked on performance in both userspace and the kernel and I think you’re fundamentally way off-base in a way we’ll never reconcile.

Re: Fixing stutters in Papers Please on Linux

#116

Post by treis » Sun, Jan 02, 2022, 9:14 PM UTC

CyberRabbi wrote: post #72

touisteur wrote: post #63

Earlier quoted context omitted.

Or, io_uring the thing. One could probably wrap close() with LD_PRELOAD and not touch the binary...

While tempting, you can’t generally fix this by simply patching close() with some function that converts it to an unchecked asynchronous operation. If that were the case, you could just do that in the kernel. Close() is expected to complete synchronously. This matters because posix guarantees that open()/pipe() etc. will return the lowest file descriptor not in use[1]. I.e. this should work: close(0); fd = open(“/foo…

>This matters because posix guarantees that open()/pipe() etc. will return the lowest file descriptor not in use[1]. I.e. this should work: close(0); fd = open(“/foo/bar”, …); // fd is guaranteed to be 0

On a multi threaded system that isn't guaranteed is it? Meaning, another thread could call open in-between your close & open.

Re: Fixing stutters in Papers Please on Linux

#117

Post by loeg » Sun, Jan 02, 2022, 9:15 PM UTC

dagmx wrote: post #96

loeg wrote: post #89

Earlier quoted context omitted.

> Blocking simply means that the specification does not guarantee an upper bound on the completion time. I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking. > I think this is an instance of confusing what should be with what is. Who is doing the confusing? I said "should be." Are you saying they're fast now but should be slow? Why? > The reality is…

CyberRabbi's definition of blocking is correct and what I've always seen commonly accepted. Blocking means you don't know how long it'll take, and you want to wait for it to finish. The only safe assumption is that you cannot guarantee how long it'll take. getpid is accurately therefore a blocking call. You don't know how long it'll take. You can profile and make best guesses, but you can never assuredly say how long…

Every operation in a non-RTOS is blocking by this definition, even local function calls that don’t enter the kernel, because the kernel may switch to another thread at any time. It’s utterly useless as a definition. Much more common is to divide system calls into ones that call depend on some external actor and those that don’t. Eg, recv() on a socket, blocking on a futex held by some other process, or waiting on IO to some disk controller. Getpid() is synchronous but does not block.

Re: Fixing stutters in Papers Please on Linux

#118

Post by CyberRabbi » Sun, Jan 02, 2022, 9:26 PM UTC

treis wrote: post #116

CyberRabbi wrote: post #72

Earlier quoted context omitted.

While tempting, you can’t generally fix this by simply patching close() with some function that converts it to an unchecked asynchronous operation. If that were the case, you could just do that in the kernel. Close() is expected to complete synchronously. This matters because posix guarantees that open()/pipe() etc. will return the lowest file descriptor not in use[1]. I.e. this should work: close(0); fd = open(“/foo…

>This matters because posix guarantees that open()/pipe() etc. will return the lowest file descriptor not in use[1]. I.e. this should work: close(0); fd = open(“/foo/bar”, …); // fd is guaranteed to be 0 On a multi threaded system that isn't guaranteed is it? Meaning, another thread could call open in-between your close & open.

It is guaranteed whether multi-threaded or not. It’s a process level guarantee. If your application is designed such that you don’t know what your other threads are doing then POSIX cannot help you.

Re: Fixing stutters in Papers Please on Linux

#119

Post by CyberRabbi » Sun, Jan 02, 2022, 9:32 PM UTC

loeg wrote: post #117

dagmx wrote: post #96

Earlier quoted context omitted.

CyberRabbi's definition of blocking is correct and what I've always seen commonly accepted. Blocking means you don't know how long it'll take, and you want to wait for it to finish. The only safe assumption is that you cannot guarantee how long it'll take. getpid is accurately therefore a blocking call. You don't know how long it'll take. You can profile and make best guesses, but you can never assuredly say how long…

Every operation in a non-RTOS is blocking by this definition, even local function calls that don’t enter the kernel, because the kernel may switch to another thread at any time. It’s utterly useless as a definition. Much more common is to divide system calls into ones that call depend on some external actor and those that don’t. Eg, recv() on a socket, blocking on a futex held by some other process, or waiting on IO…

Blocking in that sense is usually used in relation to some event. E.g. sleep() blocks on a timer, read() blocks on IO, etc.

In the general sense, it means that the call has an indefinite run time. E.g. “this call blocks” = “this call could take an arbitrarily long amount of time”

getpid() is blocking but it likely does not block on IO (though it could as that is allowed by the spec).

Re: Fixing stutters in Papers Please on Linux

#120

Post by CyberRabbi » Sun, Jan 02, 2022, 9:40 PM UTC

loeg wrote: post #115

CyberRabbi wrote: post #90

Earlier quoted context omitted.

> I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking. When it comes to expecting a specific duration, getpid() is blocking. If you run getpid() in a tight loop and then have performance issues you can’t reasonably blame the system. > This isn't a portable program; it's a Linux program But the interface is a portable interface > POSIX does not manda…

> Today the issue was close() just happened to be too “slow.” If the amount of input devices were higher, let’s say 2x more, then the same issue would have manifested even if close() were 2x “faster.” No matter how fast you make close() there is a situation in which this issue would manifest itself. Close, on an fd for which no asynchronous IO has occurred, should be 10000x faster, or more. It’s unlikely a user will…

> I agree the algorithm leaves something to be desired, but the only reason it is user-visible is the performance bug in Linux.

The only reason it wasn’t user-visible was luck. Robust applications don’t depend on luck.

Something tells me you’ll think twice before calling close() in a time-sensitive context in your future performance engineering endeavors. That’s because both you and I now know that no implementation of POSIX makes any guarantee on the runtime of close() nor will likely do so in the future. That’s just reality kicking in. Welcome to the club :)