Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fixing stutters in Papers Please on Linux

blog.jhm.dev

Post reply on HN
Print view View original

141–150 of 202 posts

Re: Fixing stutters in Papers Please on Linux

#141

Post by toast0 »

R0b0t1 wrote: post #140
CyberRabbi wrote: post #72

Earlier quoted context omitted.

While tempting, you can’t generally fix this by simply patching close() with some function that converts it to an unchecked asynchronous operation. If that were the case, you could just do that in the kernel. Close() is expected to complete synchronously. This matters because posix guarantees that open()/pipe() etc. will return the lowest file descriptor not in use[1]. I.e. this should work: close(0); fd = open(“/foo…

Are you sure such code exists? Doesn't the standard tell you to always treat the fd type as opaque anyway? Referring to exactly the point you cite, the standard seems to be making no strong statement at all. It says to allocate from the lowest fd but that calls which may return multiple fds do not need to guarantee they are adjacent. I always took this to mean the values should pack downward and should not be e.g. al…

> says to allocate from the lowest fd but that calls which may return multiple fds do not need to guarantee they are adjacent.

If the program has fd 0-3 and 5 open, socketpair should return 4 and 6, which are not adjacent. If socketpair is called again, while close(N) (N < 7) is being called in a separate thread, you could get {7, 8}, {N, 7}, or {7, N}, depending on kernel and timing details. All of those returns fit the requirement that the fds be allocated lowest first, but may or may not be adjacent or in absolute order.

Re: Fixing stutters in Papers Please on Linux

#142

Post by CyberRabbi »

R0b0t1 wrote: post #140
CyberRabbi wrote: post #72

Earlier quoted context omitted.

While tempting, you can’t generally fix this by simply patching close() with some function that converts it to an unchecked asynchronous operation. If that were the case, you could just do that in the kernel. Close() is expected to complete synchronously. This matters because posix guarantees that open()/pipe() etc. will return the lowest file descriptor not in use[1]. I.e. this should work: close(0); fd = open(“/foo…

Are you sure such code exists? Doesn't the standard tell you to always treat the fd type as opaque anyway? Referring to exactly the point you cite, the standard seems to be making no strong statement at all. It says to allocate from the lowest fd but that calls which may return multiple fds do not need to guarantee they are adjacent. I always took this to mean the values should pack downward and should not be e.g. al…

> I always took this to mean the values should pack downward and should not be e.g. allocated randomly, though it never seemed clear to me why,

The reason for this requirement is that early versions of Unix did not have dup2(), only dup(). It has nothing to do with multi threading as this predates pthreads by more than two decades. The shell (sh) makes use of the lowest numbered property to redirect standard in/out/error when setting up pipelines:

    int pipes[2];
    /* ignore errors */
    (void) pipe(&pipes);
    if (fork()) {
        close(0);
        /* guaranteed to return 0 */
        (void) dup(pipes[0]);
        close(pipes[0]);
        close(pipes[1]);
        exec_child();
    } else {
        close(1);
        /* guaranteed to return 1, we know 0 is taken */
        (void) dup(pipes[1]);
        close(pipes[0]);
        close(pipes[1]);
        exec_parent();
    }
Code like this exists in literally every POSIX shell. Anyone saying code like this isn’t common has no idea what they’re talking about.

Re: Fixing stutters in Papers Please on Linux

#143

Post by Dylan16807 »

CyberRabbi wrote: post #107
evouga wrote: post #101

Earlier quoted context omitted.

> The reality is that system calls aren’t “fast” and they can’t portably or dynamically be guaranteed to be fast. Perhaps, but the reality is also that the vast majority of games and other interactive applications routinely make blocking system calls in a tight main loop and expect these calls to take an unspecified but reasonable amount of time. “It’s a blocking syscall so if it takes 1s to close a file, that’s tech…

> “It’s a blocking syscall so if it takes 1s to close a file, that’s technically not a bug” is correct, but is any player of “Papers, Please” going to be sympathetic to that explanation? Probably not; they’ll think “Linux is slow,” “Linux is buggy,” “why can’t Linux run basic applications correctly that I have no problem running on Windows or OS X?,” etc. I don’t agree with this logic. Windows and macOS system calls…

> Linux already operates by this principle. We are examining a situation where best effort was not good enough to hide poor application design.

Linux has this principle as a goal, but it's probably not checked often.

I would say this code fails the principle, independent of particular application problems.

Re: Fixing stutters in Papers Please on Linux

#144

Post by Dylan16807 »

dagmx wrote: post #121
loeg wrote: post #117

Earlier quoted context omitted.

Every operation in a non-RTOS is blocking by this definition, even local function calls that don’t enter the kernel, because the kernel may switch to another thread at any time. It’s utterly useless as a definition. Much more common is to divide system calls into ones that call depend on some external actor and those that don’t. Eg, recv() on a socket, blocking on a futex held by some other process, or waiting on IO…

If you call getpid, or even local functions, can the rest of your code (in a single thread) continue till getpid returns? E.g if you do this inside a function (useless code) int pid = getpid(); std::cout Will the output print even if the hypothetical call to getpid takes a second? If the answer is the print will wait, then it's a blocking call. If it was an async call, then it could happen concurrently or in parallel…

But with such an extreme definition, can you even show me what an an async non-blocking syscall would look like?

Because I'm going to point at the assembly instructions that pass the parameters, and say "an interrupt happens here, delaying it for 1 second".

Any definition of blocking that includes "int fifty() {return 50;}" strikes me as having problems.

More specifically, I'd say there's some amount of "kernel does a thing" that needs to be excusable when you're talking about whether a syscall is blocking or not, otherwise everything is blocking.

Unless we want to say that 'nonblocking' is fake on non-RTOS systems, and not even try to define the term in that context.

Re: Fixing stutters in Papers Please on Linux

#145

Post by [deleted] »

spijdar wrote: post #128
CorrectHorseBat wrote: post #125

Earlier quoted context omitted.

It's not, but it really has come a far way and I'm extremely impressed. I'm kind of the other way around, I've never been more than a very casual gamer and I'm simply not interested in keeping a separate Windows pc or dual boot install for games. If I can't get it working on Linux I'm not bothering with it. Right now I can play any game I want to play with very minimal tinkering (that probably says more about me than…

Linux has absolutely come very far, don't get me wrong! I'm also mostly a casual gamer, and only have my "dedicated gaming PC" because it's 7 year old hardware I've replaced with a dedicated "workstation" I bought after getting a job and saving some money. On all my other hardware, I just run Linux, and I pretty much do the same as you -- most of my games work fine on Linux, a surprising number natively! Linus brough…

[deleted]

Re: Fixing stutters in Papers Please on Linux

#146

Post by Schnitz »

spijdar wrote: post #128
CorrectHorseBat wrote: post #125

Earlier quoted context omitted.

It's not, but it really has come a far way and I'm extremely impressed. I'm kind of the other way around, I've never been more than a very casual gamer and I'm simply not interested in keeping a separate Windows pc or dual boot install for games. If I can't get it working on Linux I'm not bothering with it. Right now I can play any game I want to play with very minimal tinkering (that probably says more about me than…

Linux has absolutely come very far, don't get me wrong! I'm also mostly a casual gamer, and only have my "dedicated gaming PC" because it's 7 year old hardware I've replaced with a dedicated "workstation" I bought after getting a job and saving some money. On all my other hardware, I just run Linux, and I pretty much do the same as you -- most of my games work fine on Linux, a surprising number natively! Linus brough…

Where does this sentiment come from that Linux has come very far when it comes to gaming? When Doom 3 released in 2004 I had to use a hex editor to hand patch the executable to get sound working. Luckily someone did what OP did and posted the instructions on a forum. I've used Linux/Unix for 20+ years but I wouldn't recommend it for gaming unless you enjoy debugging Linux software and want to do more of it. Frankly, you can learn a ton by doing so, it's not a waste of time, but priorities and frustration tolerances change as people get older. Then the demographic that no longer wants to debug their games tends to also be the cohort that has more money and is more willing to spend for a better experience, which means more money is being allocated to the platforms they are on as opposed to Linux.

Re: Fixing stutters in Papers Please on Linux

#147

Post by dagmx »

Dylan16807 wrote: post #144
dagmx wrote: post #121

Earlier quoted context omitted.

If you call getpid, or even local functions, can the rest of your code (in a single thread) continue till getpid returns? E.g if you do this inside a function (useless code) int pid = getpid(); std::cout Will the output print even if the hypothetical call to getpid takes a second? If the answer is the print will wait, then it's a blocking call. If it was an async call, then it could happen concurrently or in parallel…

But with such an extreme definition, can you even show me what an an async non-blocking syscall would look like? Because I'm going to point at the assembly instructions that pass the parameters, and say "an interrupt happens here, delaying it for 1 second". Any definition of blocking that includes "int fifty() {return 50;}" strikes me as having problems. More specifically, I'd say there's some amount of "kernel does…

There are two points that I've made a couple times that are perhaps getting lost:

1. It's about blocking your logic flow, not about how the system is actually executing it or what the machine code resolves to. If a subsequent call is blocked on a previous one, then it's blocking. Spawning an async function or creating a new thread etc can be blocking, whereas what runs on it isn't (for your current thread).

2. Being blocking or not is independent of performance. A blocking function call can be near instant, it may get inlined, it may take a year to run. Similarly an async or non blocking call can also have the same time complexity. The issue is that if the spec doesn't say it returns instantly, or you don't know for sure that it does, you can't guarantee that the blocking time will be short enough to be acceptable. So while getpid or close will almost always return instantly, it's still blocking. And if the spec doesn't say it's guaranteed, then the performance acceptability in the hot path can change.

End of the day it's all just (often pedantic) semantics to let people describe the execution nature of things so devs can make the best decisions for their performance needs.

Re: Fixing stutters in Papers Please on Linux

#148

Post by Dylan16807 »

dagmx wrote: post #147
Dylan16807 wrote: post #144

Earlier quoted context omitted.

But with such an extreme definition, can you even show me what an an async non-blocking syscall would look like? Because I'm going to point at the assembly instructions that pass the parameters, and say "an interrupt happens here, delaying it for 1 second". Any definition of blocking that includes "int fifty() {return 50;}" strikes me as having problems. More specifically, I'd say there's some amount of "kernel does…

There are two points that I've made a couple times that are perhaps getting lost: 1. It's about blocking your logic flow, not about how the system is actually executing it or what the machine code resolves to. If a subsequent call is blocked on a previous one, then it's blocking. Spawning an async function or creating a new thread etc can be blocking, whereas what runs on it isn't (for your current thread). 2. Being…

I think you replied before I added 'Unless we want to say that 'nonblocking' is fake on non-RTOS systems, and not even try to define the term in that context. "

Sure, the spec doesn't give a guarantee. But let's say it's impossible to give a guarantee on Linux. Is it really the best option to give up on defining 'nonblocking' entirely? Maybe we should formulate guarantees with an escape hatch for non-RTOS hazards. If we can do that, then getpid deserves one of those conditional guarantees.

And since I'm pretty sure the intent of mentioning getpid was to talk about the code, not the documentation, I think that would make it nonblocking.

> End of the day it's all just (often pedantic) semantics to let people describe the execution nature of things so devs can make the best decisions for their performance needs.

Which is why you don't want to label everything blocking. Nobody can have a useful discussion then.

And also why it's useful to talk about the execution nature of code, even when no spec exists. You don't want to get stuck on implementation details but you shouldn't ignore implementation either.

Edit:

> Spawning an async function or creating a new thread etc can be blocking, whereas what runs on it isn't (for your current thread).

There's some value in talking about functions that way, but for a syscall in particular you need a nonblocking spawn for the syscall to be nonblocking. If that's definitionally impossible, then something bad has happened to the definitions being used.

Re: Fixing stutters in Papers Please on Linux

#149

Post by slavik81 »

CyberRabbi wrote: post #134
slavik81 wrote: post #126

Earlier quoted context omitted.

There's no guarantee for the runtime of any function. It's perfectly valid for the OS to swap your program instructions to disk, and then take seconds or even minutes to load it back. It's effectively impossible to avoid depending on what you call "luck". The OS does not provide nearly enough guarantees to build useful interactive applications without also depending on other reasonable performance expectations.

> It's perfectly valid for the OS to swap your program instructions to disk, and then take seconds or even minutes to load it back. It’s not valid to swap your program instructions to disk if you call mlock() on your executable pages. Indeed, performance sensitive applications do just that. https://man7.org/linux/man-pages/man2/mlock.2.html > It's effectively impossible to avoid depending on what you call "luck". The…

Does mlock itself have a guaranteed maximum execution time? Is it guaranteed to return success under the relevant conditions? While that is an excellent way to address the problem I mentioned, you still have to depend on more than just the guaranteed behaviour of the OS.

> You likely wrote your comment on a POSIX-based interactive application. It just takes knowledge of how the system works and what the specifications are.

I wrote my comment on an interactive POSIX application, yes, but I believe my browser depends on "reasonable performance" of OS-provided functions in order to be usable.

It would be a fun exercise to evaluate such a program that supposedly did not. For any given program, I suspect I could patch the Linux kernel in such a way that the kernel still fulfilled all guaranteed behaviour while still making the program unusable.

Re: Fixing stutters in Papers Please on Linux

#150

Post by japhib »

Interesting tidbit about Papers, Please: it's written with Haxe (a programming language), on top of OpenFL (an open-source Flash alternative), which uses Lime as its foundational cross-platform library. The nice stack traces you see in the article, such as:

openfl::display::Application_obj::__construct

are because Haxe actually compiles down to C++! (That's also why it has nice interop with C/C++ libraries like SDL.) Haxe boasts an insane amount of compile targets -- just taking a glance at its Github page, it can compile to Javascript, C#, C++, Java, Lua, PHP, Python, and Flash, plus a couple different Haxe-specific interpreters.

Not many game developers use Haxe, so it has a small, tight-knit community. Other well-known games written with Haxe include Dead Cells, Northgard, and Dicey Dungeons.

Post reply on HN