Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fixing stutters in Papers Please on Linux

blog.jhm.dev

Post reply on HN
Print view View original

81–90 of 202 posts

Re: Fixing stutters in Papers Please on Linux

#81

Post by loeg »

CyberRabbi wrote: post #77
londons_explore wrote: post #76

Earlier quoted context omitted.

close() is typically a blocking operation. But when it happens in devfs, procfs, tmpfs, or some other ram only filesystem I expect it to be fast unless documented otherwise.

> I expect it to be fast unless documented otherwise. Logically you should expect it to block indefinitely unless documented otherwise. The exception would be completing within a time bound, the rule is blocking indefinitely.

> Logically you should expect it to block indefinitely

Frankly, that’s completely insane. It should block if and only if there is actual io in flight which could produce a failure return that an application needs. Syscalls should be fast unless there is a very good reason not to be.

Re: Fixing stutters in Papers Please on Linux

#82

Post by loeg »

londons_explore wrote: post #78
tankenmate wrote: post #19

Earlier quoted context omitted.

I just did a quick check the posted fix is not in the most recent -rc branch in the public git repo.

This is the issue with using mailing lists... Large numbers of perfectly good fixes, embodying many hours of effort, just get missed and forgotten about. At least with GitHub PR's, every request either needs to be merged or rejected.

I’m no fan of mailing lists, but GitHub PRs get ignored in much the same way.

Re: Fixing stutters in Papers Please on Linux

#84

Post by CyberRabbi »

loeg wrote: post #81
CyberRabbi wrote: post #77

Earlier quoted context omitted.

> I expect it to be fast unless documented otherwise. Logically you should expect it to block indefinitely unless documented otherwise. The exception would be completing within a time bound, the rule is blocking indefinitely.

> Logically you should expect it to block indefinitely Frankly, that’s completely insane. It should block if and only if there is actual io in flight which could produce a failure return that an application needs. Syscalls should be fast unless there is a very good reason not to be.

> It should block if and only if there is actual io in flight which could produce a failure return that an application needs.

Blocking simply means that the specification does not guarantee an upper bound on the completion time. There is no other meaningful definition. POSIX is not an RTOS therefore nearly all system calls block. The alternative is that the specification guarantees an upper bound on completion time. In that case what is an acceptable upper bound for close() to complete in? 1ms? 10ms? 100ms? Any answer diminishes the versatility of the POSIX VFS.

> Syscalls should be fast unless there is a very good reason not to be.

I think this is an instance of confusing what should be with what is. We’ve been through this before with O_PONIES. The reality is that system calls aren’t “fast” and they can’t portably or dynamically be guaranteed to be fast. So far the only exception to this is gettimeofday() and friends.

Robust systems aren’t built on undocumented assumptions. Again, POSIX is not an RTOS. Anything you build that assumes a deterministic upper bound to a blocking system call execution time will inevitably break, evidenced by OP.

Re: Fixing stutters in Papers Please on Linux

#85

Post by elevader »

yjftsjthsd-h wrote: post #75
vlovich123 wrote: post #53

Earlier quoted context omitted.

You could. But the amount of time it takes is significantly more than yay -S or emerge vs the hell that this poses on Debian (not to mention the dependency hell you can run into)

I don't know the situation on Gentoo, but partial updates are explicitly unsupported on Arch, not least because they don't do stable ABIs; Debian should have a much easier time upgrading just one package.

It's not necessarily recommended to mix stable and testing but it mostly works fine in my experience. I'd guess Gentoo gets around quite a few problems as everything is compiled from source. So updating a single libary would cause a rebuild of everything that depends on it.

Gentoo also has the concept of "Slots", so you could have multiple versions of the same libary installed and packages will choose their version to build against accordingly.

Re: Fixing stutters in Papers Please on Linux

#87

Post by mgaunard »

loeg wrote: post #70
Traubenfuchs wrote: post #35

Why would you not let another thread do this nasty kind of polling and let the main loop check for a changed result, if at all?

Close, on a synthetic fd with no I/O performed, should not take 100ms per call. This is a Linux performance bug.

close can take arbitrarily long, it's a blocking operation.

Don't ever call close on the hot path.

Re: Fixing stutters in Papers Please on Linux

#88

Post by virtue3 »

CyberRabbi wrote: post #84
loeg wrote: post #81

Earlier quoted context omitted.

> Logically you should expect it to block indefinitely Frankly, that’s completely insane. It should block if and only if there is actual io in flight which could produce a failure return that an application needs. Syscalls should be fast unless there is a very good reason not to be.

> It should block if and only if there is actual io in flight which could produce a failure return that an application needs. Blocking simply means that the specification does not guarantee an upper bound on the completion time. There is no other meaningful definition. POSIX is not an RTOS therefore nearly all system calls block. The alternative is that the specification guarantees an upper bound on completion time.…

Very similar to people using node.getenv in hot sections of code and the resulting not understanding what's happening.

https://github.com/nodejs/node/issues/3104

When you call out to the sys or libc things are going to happen and you should try and be aware of what those are.

Re: Fixing stutters in Papers Please on Linux

#89

Post by loeg »

CyberRabbi wrote: post #84
loeg wrote: post #81

Earlier quoted context omitted.

> Logically you should expect it to block indefinitely Frankly, that’s completely insane. It should block if and only if there is actual io in flight which could produce a failure return that an application needs. Syscalls should be fast unless there is a very good reason not to be.

> It should block if and only if there is actual io in flight which could produce a failure return that an application needs. Blocking simply means that the specification does not guarantee an upper bound on the completion time. There is no other meaningful definition. POSIX is not an RTOS therefore nearly all system calls block. The alternative is that the specification guarantees an upper bound on completion time.…

> Blocking simply means that the specification does not guarantee an upper bound on the completion time.

I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking.

> I think this is an instance of confusing what should be with what is.

Who is doing the confusing? I said "should be." Are you saying they're fast now but should be slow? Why?

> The reality is that system calls aren’t “fast” and they can’t portably or dynamically be guaranteed to be fast.

This isn't a portable program; it's a Linux program. The problem isn't that close can't be portably guaranteed to complete in some time bound; it's that Linux is adding what is essentially an extra usleep(100000), with very high probability, for the devfs synthetic filesystem in Linux.

This is entirely an own-goal; Linux has historically explicitly aimed to complete system calls quickly, when that does not break other functionality. It is a bug that can be fixed, e.g., with the proposed patch(es).

POSIX does not mandate that close blocks on anything other than removing the index from the fd table -- it's even allowed to leave associated IO in-flight and silently ignore errors. It makes little sense for a synthetic filesystem without real IO to block close so grossly.

Re: Fixing stutters in Papers Please on Linux

#90

Post by CyberRabbi »

loeg wrote: post #89
CyberRabbi wrote: post #84

Earlier quoted context omitted.

> It should block if and only if there is actual io in flight which could produce a failure return that an application needs. Blocking simply means that the specification does not guarantee an upper bound on the completion time. There is no other meaningful definition. POSIX is not an RTOS therefore nearly all system calls block. The alternative is that the specification guarantees an upper bound on completion time.…

> Blocking simply means that the specification does not guarantee an upper bound on the completion time. I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking. > I think this is an instance of confusing what should be with what is. Who is doing the confusing? I said "should be." Are you saying they're fast now but should be slow? Why? > The reality is…

> I don't think that's a commonly-accepted (or useful) definition of "blocking." By that definition, getpid(2) is blocking.

When it comes to expecting a specific duration, getpid() is blocking. If you run getpid() in a tight loop and then have performance issues you can’t reasonably blame the system.

> This isn't a portable program; it's a Linux program

But the interface is a portable interface

> POSIX does not mandate that close blocks on anything other than removing the index from the fd table

And what if the fd-table is a very large hash table with high collision rate? How do you then specify how quickly close() should complete? 1ms/open fd? 10ms/open fd? Etc.

It should be clear that the problem here is that the author of the code had a faulty understanding of the system in which their code runs. Today the issue was close() just happened to be too “slow.” If the amount of input devices were higher, let’s say 2x more, then the same issue would have manifested even if close() were 2x “faster.” No matter how fast you make close() there is a situation in which this issue would manifest itself. I.e. the application has a design flaw.

Post reply on HN