Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fixing stutters in Papers Please on Linux

blog.jhm.dev

Post reply on HN
Print view View original

191–200 of 202 posts

Re: Fixing stutters in Papers Please on Linux

#191

Post by CyberRabbi »

Dylan16807 wrote: post #190
CyberRabbi wrote: post #189

Earlier quoted context omitted.

> What resource? I did my best to look at the implementation, but the source code is complicated and scattered. I can't really process your link by itself. How often are these things causing delays? A signal may need to be invoked and that could cause paging to disk. The point is that the kernel is allowed to do a non-predictable amount of work on most system calls and therefore you cannot assume getpid() completes i…

People are going to keep using non-realtime systems to run soft realtime UIs. We can't make them stop, so it's still important to distinguish between "this syscall might hit a signal or an interrupt, just like every single line of code in the program" and "this syscall might hit a signal or an interrupt, but also it might get stuck waiting on a resource in a way that couldn't have otherwise happened". If you want to…

> People are going to keep using non-realtime systems to run soft realtime UIs.

Very true and if they want their applications to work well they should write they applications correctly!

Re: Fixing stutters in Papers Please on Linux

#192

Post by Dylan16807 »

CyberRabbi wrote: post #191
Dylan16807 wrote: post #190

Earlier quoted context omitted.

People are going to keep using non-realtime systems to run soft realtime UIs. We can't make them stop, so it's still important to distinguish between "this syscall might hit a signal or an interrupt, just like every single line of code in the program" and "this syscall might hit a signal or an interrupt, but also it might get stuck waiting on a resource in a way that couldn't have otherwise happened". If you want to…

> People are going to keep using non-realtime systems to run soft realtime UIs. Very true and if they want their applications to work well they should write they applications correctly!

The best way to help them write applications correctly is not to say "all syscalls are blocking, none are nonblocking, no other categories".

Re: Fixing stutters in Papers Please on Linux

#193

Post by CyberRabbi »

Dylan16807 wrote: post #192
CyberRabbi wrote: post #191

Earlier quoted context omitted.

> People are going to keep using non-realtime systems to run soft realtime UIs. Very true and if they want their applications to work well they should write they applications correctly!

The best way to help them write applications correctly is not to say "all syscalls are blocking, none are nonblocking, no other categories".

There are categories, some system calls block on timers, some block on disk io, some block on network io. But they all block, except for gettimeofday() and friends.

Re: Fixing stutters in Papers Please on Linux

#194

Post by utbabya »

CyberRabbi wrote: post #120
loeg wrote: post #115

Earlier quoted context omitted.

> Today the issue was close() just happened to be too “slow.” If the amount of input devices were higher, let’s say 2x more, then the same issue would have manifested even if close() were 2x “faster.” No matter how fast you make close() there is a situation in which this issue would manifest itself. Close, on an fd for which no asynchronous IO has occurred, should be 10000x faster, or more. It’s unlikely a user will…

> I agree the algorithm leaves something to be desired, but the only reason it is user-visible is the performance bug in Linux. The only reason it wasn’t user-visible was luck. Robust applications don’t depend on luck. Something tells me you’ll think twice before calling close() in a time-sensitive context in your future performance engineering endeavors. That’s because both you and I now know that no implementation…

I agree the application should not have done this. On the other hand I also agree indefinite block time is not a useful definition despite being correct in theory, perhaps a more pragmatic one would be some time / compute unit percentile? So a consistent 100ms close call which is proven to be a bug won't get lost in definition.

Re: Fixing stutters in Papers Please on Linux

#195

Post by Dylan16807 »

CyberRabbi wrote: post #193
Dylan16807 wrote: post #192

Earlier quoted context omitted.

The best way to help them write applications correctly is not to say "all syscalls are blocking, none are nonblocking, no other categories".

There are categories, some system calls block on timers, some block on disk io, some block on network io. But they all block, except for gettimeofday() and friends.

I mean I wouldn't say gettimeofday is significantly better than getpid because your thread might switch out anyway. But sure five categories is fine, I just dislike lumping almost everything together.

Re: Fixing stutters in Papers Please on Linux

#196

Post by kennylevinsen »

CyberRabbi wrote: post #176
kennylevinsen wrote: post #167

Earlier quoted context omitted.

Patch author here. It is important to not conflate POSIX requirements with expected behavior, especially for device files which require very specific knowledge of their implementation to use (DRM ioctl's and resources anyone?). You might think that as a well-behaved game should not be opening/closing evdev fds during gameplay at all, this is clearly just an application bug. However, games are not the main user of evd…

> As for putting things in threads, I would consider it a huge hack to move open/close. Threads are not and will never be mandatory to have great responsiveness. The POSIX interface was invented for batch processing. Long running non-interactive jobs. This is why it lacks timing requirements. All well-designed interactive GUI applications do not interact with the file system on their main thread. This is especially t…

> All well-designed interactive GUI applications do not interact with the file system on their main thread

I strongly disagree. A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time.

The POSIX interfaces provide sufficient non-blocking functionality for this to be true, and the (as per the documentation, "brief") blocking allowed by things like open/close is not an issue.

(io_uring is still a nice improvement though.)

> I don’t think that is proven unless the original author can chime in.

This argument is nonsense. Whether or not code is buggy does not depend on whether or not the author comments on the matter. This is especially true for a project as vast as the Linux kernel with its massive number of ever-changing authors.

> If it is indeed the case the synchronize_rcu is taking up to 50ms I would suspect there is a deeper issue at play on this machine. By search/replacing the call with call_rcu or similar you may just be masking the problem. RCU updates should not be taking that long.

synchronize_rcu is designed to block for a significant amount of time, but I did not push the patch further exactly because I would like to dig deeper into the issue rather than making a text-book RCU fix.

Re: Fixing stutters in Papers Please on Linux

#197

Post by CyberRabbi »

kennylevinsen wrote: post #196
CyberRabbi wrote: post #176

Earlier quoted context omitted.

> As for putting things in threads, I would consider it a huge hack to move open/close. Threads are not and will never be mandatory to have great responsiveness. The POSIX interface was invented for batch processing. Long running non-interactive jobs. This is why it lacks timing requirements. All well-designed interactive GUI applications do not interact with the file system on their main thread. This is especially t…

> All well-designed interactive GUI applications do not interact with the file system on their main thread I strongly disagree. A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time. The POSIX interfaces provide sufficient non-blocking functionality for this to be t…

> A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time.

Hmm. If you call open()/read()/close() on the main thread and it causes a high latency network operation because that user happens to have their home directory on a network file system like NFS or SMB, your application will appear to hang. When you design applications you can’t just assume your users have the same setup as you.

> The POSIX interfaces provide sufficient non-blocking functionality for this to be true

POSIX file system IO is always blocking, even with O_NONBLOCK. You can use something like io_uring to do non blocking file system io but that would no longer be POSIX.

> Whether or not code is buggy does not depend on whether or not the author comments on the matter.

That would depend on if you knew more about how the code is intended to work than the original author of the code. Do you presume to know more about how this code is intended to work than the original author?

Re: Fixing stutters in Papers Please on Linux

#198

Post by diamondlovesyou »

kennylevinsen wrote: post #196
CyberRabbi wrote: post #176

Earlier quoted context omitted.

> As for putting things in threads, I would consider it a huge hack to move open/close. Threads are not and will never be mandatory to have great responsiveness. The POSIX interface was invented for batch processing. Long running non-interactive jobs. This is why it lacks timing requirements. All well-designed interactive GUI applications do not interact with the file system on their main thread. This is especially t…

> All well-designed interactive GUI applications do not interact with the file system on their main thread I strongly disagree. A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time. The POSIX interfaces provide sufficient non-blocking functionality for this to be t…

> A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time.

The "well-designed" argument here is a bit No True Scotsman, and absolutely not true. Consider a lagging NFS mount. Or old hard drives; a disk seek could take milliseconds!

Real time computing isn't about what is normal or average, it's about the worst case. Filesystem IO can block, therefore you must assume it will.

Re: Fixing stutters in Papers Please on Linux

#199

Post by kennylevinsen »

diamondlovesyou wrote: post #198
kennylevinsen wrote: post #196

Earlier quoted context omitted.

> All well-designed interactive GUI applications do not interact with the file system on their main thread I strongly disagree. A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time. The POSIX interfaces provide sufficient non-blocking functionality for this to be t…

> A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time. The "well-designed" argument here is a bit No True Scotsman, and absolutely not true. Consider a lagging NFS mount. Or old hard drives; a disk seek could take milliseconds! Real time computing isn't about what…

> The "well-designed" argument here is a bit No True Scotsman, and absolutely not true.

This counter arguments can be interpreted as a mere No True Scotsman of "responsiveness", so this is not a very productive line of argument.

Should one be interested in having a discussion like this again, I would suggest strictly establishing what "responsive" means (which is a subjective experience), including defining when a "responsive" application may be "unresponsive" (swapping to disk, no CPU/GPU time, the cat ate the RAM), and evading terms like "well-designed" (I included it in protest of its use in the comment I responded to).

For example, failing to process input or skipping frames in gameplay would be bad, but no one would see a skipped frame in a config menu, and frames cannot even be skipped if there are no frames to be rendered.

Re: Fixing stutters in Papers Please on Linux

#200

Post by kennylevinsen »

CyberRabbi wrote: post #197
kennylevinsen wrote: post #196

Earlier quoted context omitted.

> All well-designed interactive GUI applications do not interact with the file system on their main thread I strongly disagree. A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time. The POSIX interfaces provide sufficient non-blocking functionality for this to be t…

> A well-designed interactive GUI application can absolutely interact with the filesystem on its main thread without any impact to responsiveness what-so-ever. You only need threads once you need more CPU time. Hmm. If you call open()/read()/close() on the main thread and it causes a high latency network operation because that user happens to have their home directory on a network file system like NFS or SMB, your ap…

> That would depend on if you knew more about how the code is intended to work than the original author of the code. Do you presume to know more about how this code is intended to work than the original author?

I am not sure if you are suggesting that only the author can know how code is supposed to work, that finding bugs require understanding of the code strictly superior to the author, or that the author is infallible and intended every behavior of the current operation.

Either way, this attitude would not have made for a healthy open source contribution environment.

Post reply on HN