Live data from Hacker News

RT64: N64 graphics renderer in emulators and native ports

github.com

41–50 of 54 posts

Re: RT64: N64 graphics renderer in emulators and native ports

#41
post #4

> Uses ubershaders to guarantee no stutters due to pipeline compilation. I may sound way out of the loop here, but... How come this was never a problem for older dx9/dx11/GL games and emulators?

I think there is some confusion about "ubershaders" in the context of emulators in particular. Old Nintendo consoles like the N64 or the GameCube/Wii didn't have programmable shaders. Instead, it was a mostly fixed-function pipeline but you could configure some stages of it to kind of somewhat fake "programmable" shaders with this configurable pipeline, at least to some degree. Now the problem is, you have no idea what any particular game is going to do, right until the moment it writes a specific configuration value into a specific GPU register, which instantly configures the GPU to do whatever the game wants it to do from that very moment onwards. There literally is no "shader" stored in the ROM, it's just code configuring (parts of) the GPU directly.

That's not how any modern GPU works though. Instead, you have to emulate this semi-fixed-function pipeline with shaders. Emulators try to generate shader code for the current GPU configuration and compile it, but that takes time and can only be done after the configuration was observed for the first time. This is where "Ubershaders" enter the scene: they are a single huge shader which implements the complete configurable semi-fixed-function pipeline, so you pass in the configuration registers to the shader and it acts accordingly. Unfortunately, such shaders are huge and slow, so you don't want to use them unless it's necessary. The idea is then to prepare "ubershaders" as fallback, use them whenever you see a new configuration, compile the real shader and cache it, and use the compiled shader once it's available instead of the ubershader, to improve performance again. A few years ago, the developers of the Dolphin emulator (GameCube/Wii) wrote an extensive blog post about how this works: https://de.dolphin-emu.org/blog/2017/07/30/ubershaders/

Only starting with the 3DS/Wii U, Nintendo consoles finally got "real" programmable shaders, in which case you "just" have to translate them to whatever you need for your host system. You still won't know which shaders you'll see until you observe the transfer of the compiled shader code to the emulated GPU. After all, the shader code is compiled ahead of time to GPU instructions, usually during the build process of the game itself. At least for Nintendo consoles, there are SDK tools to do this. This, of course, means, there is no compilation happening on the console itself, so there is no stutter caused by shader compilation either. Unlike in an emulation of such a console, which has to translate and recompile such shaders on the fly.

> How come this was never a problem for older [...] emulators?

Older emulators had highly inaccurate and/or slow GPU emulation, so this was not really a problem for a long time. Only once the GPU emulations became accurate enough with dynamically generated shaders for high performance, the shader compilation stutters became a real problem.

Re: RT64: N64 graphics renderer in emulators and native ports

#42
post #38

Here's a 2022 video that shows it in action https://www.reddit.com/r/Games/comments/v42611/dario_on_twit...

Doesn't seem to be a noticeable improvement from N64-era graphics. It probably needs generative AI based upscaling to high resolution meshes, textures and realistic materials to actually achieve a quality improvement.

The improvement is in the shadows. Link’s real-time shadow on the ground, the self-shadowing of the roof of the house that Luigi emerges from, the shadows the ropes cast on Kirby… the N64 could not do these things even at coarse resolution due to limited texture bandwidth.

Re: RT64: N64 graphics renderer in emulators and native ports

#43
post #4

> Uses ubershaders to guarantee no stutters due to pipeline compilation. I may sound way out of the loop here, but... How come this was never a problem for older dx9/dx11/GL games and emulators?

I think there is some confusion about "ubershaders" in the context of emulators in particular. Old Nintendo consoles like the N64 or the GameCube/Wii didn't have programmable shaders. Instead, it was a mostly fixed-function pipeline but you could configure some stages of it to kind of somewhat fake "programmable" shaders with this configurable pipeline, at least to some degree. Now the problem is, you have no idea wh…

> Old Nintendo consoles like the N64 or the GameCube/Wii didn't have programmable shaders.

The N64 did in fact have a fully programmable pipeline. [1] At boot, the game initialized the RSP (the N64’s GPU) with “microcode”, which was a program that implemented the RSP’s graphics pipeline. During gameplay, the game uploaded “display lists” of opcodes to the GPU which the microcode interpreted. (I misspoke earlier by referring to these opcodes as “microcode”.) For most of the console’s lifespan, game developers chose between two families of vendor-authored microcode: Fast3D and Turbo3D. Toward the end, some developers (notably Factor5) wrote their own microcode.

[1]: https://www.copetti.org/writings/consoles/nintendo-64/

Re: RT64: N64 graphics renderer in emulators and native ports

#44
post #43

Earlier quoted context omitted.

I think there is some confusion about "ubershaders" in the context of emulators in particular. Old Nintendo consoles like the N64 or the GameCube/Wii didn't have programmable shaders. Instead, it was a mostly fixed-function pipeline but you could configure some stages of it to kind of somewhat fake "programmable" shaders with this configurable pipeline, at least to some degree. Now the problem is, you have no idea wh…

> Old Nintendo consoles like the N64 or the GameCube/Wii didn't have programmable shaders. The N64 did in fact have a fully programmable pipeline. [1] At boot, the game initialized the RSP (the N64’s GPU) with “microcode”, which was a program that implemented the RSP’s graphics pipeline. During gameplay, the game uploaded “display lists” of opcodes to the GPU which the microcode interpreted. (I misspoke earlier by re…

Microcode was only used for the RSP, which was a modified MIPS coprocessor and could only realistically be used for T&L. After that, the RSP then sends triangles to the RDP for rasterization, pixel pipeline, and blending, all of which are fixed-function, admittedly with some somewhat flexible color combiner stuff.

Re: RT64: N64 graphics renderer in emulators and native ports

#45
post #14

Earlier quoted context omitted.

Compiling shaders directly from a high level representation to the GPU ISA only really happens on consoles. In DirectX on PC, shaders have been compiled into an intermediate form going back to Direct3D 8. All of these intermediate forms are lowered into an ISA-specific instruction set by the drivers. This final compilation step is triggered lazily when a draw happens, so if you are working on a "modern" engine that u…

> Compiling shaders directly from a high level representation to the GPU ISA only really happens on consoles. When targeting Apple platforms, you can use the metal-tt tool to precompile your shaders to ISA. You give it a list of target triples and a JSON file that describes your PSO. metal-tt comes with Xcode, and is also available for Windows as part of the Game Porting Toolkit. Unfortunately, most people don’t do t…

metal-tt/metal-nt require you to specify the exact architecture, and that's not forward-compatible unless you update your application for every new device release. Even minor SKU revisions like applegpu_g13p/applegpu_g13g/applegpu_g13s/applegpu_g13c are different compilation targets, and that doesn't help you when Apple releases applegpu_g14.

Re: RT64: N64 graphics renderer in emulators and native ports

#46
post #10
post #4

> Uses ubershaders to guarantee no stutters due to pipeline compilation. I may sound way out of the loop here, but... How come this was never a problem for older dx9/dx11/GL games and emulators?

It was, in fact, a problem. DX11 and earlier tried to solve it with DXBC, an intermediate bytecode format that all drivers could consume. The driver would only need to lower the bytecode to the GPU’s ISA, which is much faster than a full compilation from HLSL. Prior to the emergence of Vulkan, OpenGL didn’t ever try to solve this; GLSL was always the interface to the driver. (Nowadays SPIR-V support is available to O…

Having worked on some of the latter part of that era of GPUs, the "frontend" of the shader compiler was a pretty small fraction of the total time cost, most of it was in the later optimization passes that often extremely hardware specific (so not really possible at the level of DXBC). Especially as hardware started to move away from the assumptions used in designing it.

I think a big part of the user-visible difference in stutter is simply the expected complexity of shaders and number of different shaders in an "average" scene - they're 100s of times larger, and CPUs aren't 100s of times faster (and many of the optimization algorithms used are more-than-linear in terms of time vs the input too)

Modern DXIL and SPIR-V are at a similar level of abstraction to DXBC, and certainly don't "solve" stutter.

Re: RT64: N64 graphics renderer in emulators and native ports

#47
post #39
post #38

Earlier quoted context omitted.

Doesn't seem to be a noticeable improvement from N64-era graphics. It probably needs generative AI based upscaling to high resolution meshes, textures and realistic materials to actually achieve a quality improvement.

It would be an interesting feedback loop to capture a screenshot of a game (sm64), determine the green texture is grass, update the texture realtime, take another screenshot, and iterate, having the game flesh itself out realtime as it’s being played.

Four years ago there was demo of exactly that in GTA5 https://www.ign.com/articles/experimental-ai-tool-makes-gta-...

Re: RT64: N64 graphics renderer in emulators and native ports

#48
post #44
post #43

Earlier quoted context omitted.

> Old Nintendo consoles like the N64 or the GameCube/Wii didn't have programmable shaders. The N64 did in fact have a fully programmable pipeline. [1] At boot, the game initialized the RSP (the N64’s GPU) with “microcode”, which was a program that implemented the RSP’s graphics pipeline. During gameplay, the game uploaded “display lists” of opcodes to the GPU which the microcode interpreted. (I misspoke earlier by re…

Microcode was only used for the RSP, which was a modified MIPS coprocessor and could only realistically be used for T&L. After that, the RSP then sends triangles to the RDP for rasterization, pixel pipeline, and blending, all of which are fixed-function, admittedly with some somewhat flexible color combiner stuff.

I appreciate the correction. Still, programmable T&L was kind of a big deal. PC GPUs didn’t get hardware T&L until the DX7 era, and it didn’t really become programmable until DX9/Shader Model 2.0.

Re: RT64: N64 graphics renderer in emulators and native ports

#49
post #45
post #14

Earlier quoted context omitted.

> Compiling shaders directly from a high level representation to the GPU ISA only really happens on consoles. When targeting Apple platforms, you can use the metal-tt tool to precompile your shaders to ISA. You give it a list of target triples and a JSON file that describes your PSO. metal-tt comes with Xcode, and is also available for Windows as part of the Game Porting Toolkit. Unfortunately, most people don’t do t…

metal-tt/metal-nt require you to specify the exact architecture, and that's not forward-compatible unless you update your application for every new device release. Even minor SKU revisions like applegpu_g13p/applegpu_g13g/applegpu_g13s/applegpu_g13c are different compilation targets, and that doesn't help you when Apple releases applegpu_g14.

Yep. That’s why the artifact includes the IR, which can be compiled on first use on new architectures.

Re: RT64: N64 graphics renderer in emulators and native ports

#50
post #46
post #10

Earlier quoted context omitted.

It was, in fact, a problem. DX11 and earlier tried to solve it with DXBC, an intermediate bytecode format that all drivers could consume. The driver would only need to lower the bytecode to the GPU’s ISA, which is much faster than a full compilation from HLSL. Prior to the emergence of Vulkan, OpenGL didn’t ever try to solve this; GLSL was always the interface to the driver. (Nowadays SPIR-V support is available to O…

Having worked on some of the latter part of that era of GPUs, the "frontend" of the shader compiler was a pretty small fraction of the total time cost, most of it was in the later optimization passes that often extremely hardware specific (so not really possible at the level of DXBC). Especially as hardware started to move away from the assumptions used in designing it. I think a big part of the user-visible differen…

One advantage of contemporary bytecode implementations is that many optimizations can occur in the “middle end”—which is to say on the IR itself, before lowering to ISA.
Post reply on HN