Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

dolphin-emu.org

Post reply on HN
Print view View original

31–40 of 72 posts

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#31

Post by mappu »

naikrovek wrote: post #21
mappu wrote: post #19

Earlier quoted context omitted.

They're precompiled for the console GPU architecture, not the PC architecture, so they can't be used directly and still need to be emulated - I think those precompiled shaders are the input to the ubershader.

The GAMES THEMSELVES are precompiled for the PowerPC architecture, not the PC architecture, though. That didn’t stop anyone from creating Dolphin. GPUs (I’m told) have far fewer instructions to emulate than a CPU, so I’d think that low level emulation of the Flipper shaders would be no trouble. Can’t translate or transpile them to PC GPUs though because those instruction sets are somewhat secret, I think. I know noth…

> because those instruction sets are somewhat secret, I think

The GPU ISAs are known (e.g. the PTX compiler for NVidia is open source and has a backend in LLVM). The main problem is that the GPU ISA changes with every GPU hardware generation and manufacturer, so if you want to support Nvidia 3xxx + 4xxx + AMD VLIW + AMD GCN + ... you have to use the common demoninator GLSL/HLSL/SPIR-V/whatever.

> why they didn’t emulate Flipper at a low enough level to use the precompiled shaders directly.

They did. Originally the GPU emulator was done in the CPU, and in 2017, the GPU emulator itself was moved into a shader ("ubershader").

The console game itself does not include shaders in text format like many PC games do.

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#32

Post by db48x »

conorpo wrote: post #14

Does anyone know why this isn't an issue for modern games on PC? I assume it's because more uniforms are used, and the amount of shaders that actually need to be compiled at runtime is minimized, not to mention that the Graphics API is optimized to compile the shaders in the format they are provided. So is the issue with Dolphin that GameCube games would compile new shaders for lots of different configurations of eff…

The programmer of a modern PC game knows that they will be using shaders, and can arrange for them all to be compiled and sent to the GPU during a loading screen. That eliminates the lag, because there is no delay when choosing which shader to use for the next triangle. On the other hand it makes the loading screen take longer.

Meanwhile the programmer of a console game, not using shaders, could set GPU registers to any configuration they wanted just before rendering the next triangle. You have to actually play the game to find out what configurations it programs into the GPU, because those configurations are not neatly organized into a set of discrete shaders. Even then there is no guarantee that you found all possible configurations used by the game. The videos in the article provide a good example: the player fires a gun with luminous bullets, so on that frame the walls and floors need to be rendered with an extra light source. That requires reconfiguring the GPU to take that light source into account, then changing the configuration to render the weapon itself, then changing it again to render the HUD, and so on.

Now imagine that you go to a place on a different level where the walls are not shiny, and it doesn't bother to render the walls with the extra light source. Or it renders them with extra vertex lighting but not extra specular lighting. Now combine that with every type of wall and floor in the game; they might all need a unique shader to be lit correctly by that one gun. To find all possible GPU configurations you need to fire that gun, and every other, near every single different type of wall and floor texture used in the game. And there are a dozen different guns.

And then you need to do it all again while wearing the night–vision goggles, because that causes everything to be rendered with a different configuration yet again.

Every one of those unique combinations needs to be made into a shader, and there’s just no way to be sure that you have actually collected all of them. Or you can write a single Ubershader that can, by using branches, loops, and other advanced tricks, emulate the entire capabilities of the emulated GPU. Then you can program the Ubershader by sending all of the emulated GPUs register values as uniforms.

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#33

Post by pjmlp »

ruined wrote: post #23
conorpo wrote: post #14

Does anyone know why this isn't an issue for modern games on PC? I assume it's because more uniforms are used, and the amount of shaders that actually need to be compiled at runtime is minimized, not to mention that the Graphics API is optimized to compile the shaders in the format they are provided. So is the issue with Dolphin that GameCube games would compile new shaders for lots of different configurations of eff…

gcn games never anticipated shader compilation time because it didn't exist. shaders weren't a thing when the gcn released. it may be arguable, but nobody even used the word at the time. shader compilation time is an issue for modern games on the PC, and because of this, developers anticipate and work around it. on the gcn, specialized fixed-function pipelines were available, and could be composed by some limited con…

Shaders as concept exists since 1980's,

https://en.m.wikipedia.org/wiki/TMS34010

https://en.m.wikipedia.org/wiki/RenderMan_Shading_Language

Just two examples, SIGGRAPH has plenty of paper on the subject.

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#34

Post by pjmlp »

rtpg wrote: post #15
conorpo wrote: post #14

Does anyone know why this isn't an issue for modern games on PC? I assume it's because more uniforms are used, and the amount of shaders that actually need to be compiled at runtime is minimized, not to mention that the Graphics API is optimized to compile the shaders in the format they are provided. So is the issue with Dolphin that GameCube games would compile new shaders for lots of different configurations of eff…

I think it's because PC games know they need to deal with compilation, so they do it on the load screen or whatever. GC games can pre-compile them and just stuff it on the disk, so there's no compilation cost.

GC shaders are loosely based on GLSL, and OpenGL compilation model, even though GX(2) isn't OpenGL.

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#35

Post by pests »

naikrovek wrote: post #22
phkahler wrote: post #20

Earlier quoted context omitted.

Wait, I hought that's what the ubershaders are. What you say is what I kept thinking for much of the article - "just" emulate the GPU, no compiler needed. And then they did.

Maybe, lol, there were a lot of terms in that article that I didn’t understand well enough to claim I understood the article in toto.

One thing to remember is these older consoles don't have the same concept of a "shader" as we do today.

Go back far enough and you'll find the industry trying to settle on quads or triangles for rendering (and we all knew who won)

The games were given basically an immediate mode API into the graphics card and they could do whatever they wanted, whenever they wanted, without warning.

The stutter happened when they were translating the API mentioned above into modern GPU shaders.

When it was on the CPU - They had to determine the effect, generate and compile the modern shaders, and upload that to the GPU, sometimes hundreds of times a second. Then the GPU would take over and display.

Uber shaders took that entire pipeline and moved it into the GPU.

This was low level emulation, just still hitting limits of modern CPUs.

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#36

Post by OCASMv2 »

ruined wrote: post #23
conorpo wrote: post #14

Does anyone know why this isn't an issue for modern games on PC? I assume it's because more uniforms are used, and the amount of shaders that actually need to be compiled at runtime is minimized, not to mention that the Graphics API is optimized to compile the shaders in the format they are provided. So is the issue with Dolphin that GameCube games would compile new shaders for lots of different configurations of eff…

gcn games never anticipated shader compilation time because it didn't exist. shaders weren't a thing when the gcn released. it may be arguable, but nobody even used the word at the time. shader compilation time is an issue for modern games on the PC, and because of this, developers anticipate and work around it. on the gcn, specialized fixed-function pipelines were available, and could be composed by some limited con…

A 2002 postmortem on the tech of Rogue Leader, a Gamecube launch title, already using the term "shaders":

https://www.gamedeveloper.com/programming/shader-integration...

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#37

Post by Jasper_ »

pjmlp wrote: post #34
rtpg wrote: post #15

Earlier quoted context omitted.

I think it's because PC games know they need to deal with compilation, so they do it on the load screen or whatever. GC games can pre-compile them and just stuff it on the disk, so there's no compilation cost.

GC shaders are loosely based on GLSL, and OpenGL compilation model, even though GX(2) isn't OpenGL.

No, GameCube's TEVs have nothing to do with GLSL. The reference to GX2 implies you're confusing it with Wii U, inexplicably.

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#38

Post by doophus »

What was the missing piece for "shader sharing"?

Would it be possible to build a web-hosted database of encountered shader configs against a game id, and have Dolphin fetch that list when a game launches and start doing async compilation?

When Dolphin encounters a new shader that wasn't in the db, it phones home to request it to be added it to the list.

I feel an automated sharing solution would build up coverage pretty quickly, and finding a stutter would eventually be considered an achievement - "no-one's been here before!"

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#39

Post by pjmlp »

Jasper_ wrote: post #37
pjmlp wrote: post #34

Earlier quoted context omitted.

GC shaders are loosely based on GLSL, and OpenGL compilation model, even though GX(2) isn't OpenGL.

No, GameCube's TEVs have nothing to do with GLSL. The reference to GX2 implies you're confusing it with Wii U, inexplicably.

Yes I mixed them with Wii and Wii U,no need for "inexplicably" remark.

Re: Ubershaders: A Ridiculous Solution to an Impossible Problem (2017)

#40

Post by mandarax8 »

mappu wrote: post #31
naikrovek wrote: post #21

Earlier quoted context omitted.

The GAMES THEMSELVES are precompiled for the PowerPC architecture, not the PC architecture, though. That didn’t stop anyone from creating Dolphin. GPUs (I’m told) have far fewer instructions to emulate than a CPU, so I’d think that low level emulation of the Flipper shaders would be no trouble. Can’t translate or transpile them to PC GPUs though because those instruction sets are somewhat secret, I think. I know noth…

> because those instruction sets are somewhat secret, I think The GPU ISAs are known (e.g. the PTX compiler for NVidia is open source and has a backend in LLVM). The main problem is that the GPU ISA changes with every GPU hardware generation and manufacturer, so if you want to support Nvidia 3xxx + 4xxx + AMD VLIW + AMD GCN + ... you have to use the common demoninator GLSL/HLSL/SPIR-V/whatever. > why they didn’t emul…

> The GPU ISAs are known (e.g. the PTX compiler for NVidia is open source and has a backend in LLVM)

PTX is only and IR afaik, kinda like SPIRV. It also goes through another compiler in the driver so doesn't really help here

Post reply on HN