Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

github.com

Post reply on HN
Print view View original

31–40 of 43 posts

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#31

Post by 8n4vidtmkvmk »

jsheard wrote: post #17
8n4vidtmkvmk wrote: post #16

Earlier quoted context omitted.

I'm not convinced. We have "hyper" and "lightning" diffusion models that run 1-4 steps and are pretty quick on consumer hardware. I really have no idea which would be quicker with some optimizations and hardware tailored for the use-case.

The hard part is keeping everything coherent over time in a dynamic scene with a dynamic camera. Hallucinating vaguely plausible lighting may be adequate for a still image, but not so much in a game if you hallucinate shadows or reflections of off-screen objects that aren't really there, or "forget" that off-screen objects exist, or invent light sources that make no sense in context. The main benefit of raytracing in…

Right, but I'm not actually suggesting we use diffusion. At least, not the same models we're using now. We need to incorporate a few sample rays at least so that it 'knows' what's actually off-screen, and then we just give it lots of training data of partially rendered images and fully rendered images so that it learns how to fill in the gaps. It shouldn't hallucinate very much if we do that. I don't know how to solve for temporal coherence though -- I guess we might want to train on videos instead of still images.

Also, that new Google paper where it generates entire games from a single image has up to 60 seconds of 'memory' I think they said, so I don't think the "forgetting" is actually that big of a problem since we can refresh the memory with a properly rendered image at least every that often.

I'm just spitballing here though, I think all of Unreal 5.4 or 5.5 has put this into practice already with their new lighting system.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#32

Post by jsheard »

8n4vidtmkvmk wrote: post #31
jsheard wrote: post #17

Earlier quoted context omitted.

The hard part is keeping everything coherent over time in a dynamic scene with a dynamic camera. Hallucinating vaguely plausible lighting may be adequate for a still image, but not so much in a game if you hallucinate shadows or reflections of off-screen objects that aren't really there, or "forget" that off-screen objects exist, or invent light sources that make no sense in context. The main benefit of raytracing in…

Right, but I'm not actually suggesting we use diffusion. At least, not the same models we're using now. We need to incorporate a few sample rays at least so that it 'knows' what's actually off-screen, and then we just give it lots of training data of partially rendered images and fully rendered images so that it learns how to fill in the gaps. It shouldn't hallucinate very much if we do that. I don't know how to solv…

> We need to incorporate a few sample rays at least so that it 'knows' what's actually off-screen, and then we just give it lots of training data of partially rendered images and fully rendered images so that it learns how to fill in the gaps.

That's already a thing, there's ML-driven denoisers which take a rough raytraced image and do their best to infer what the fully converged image would look like based on their training data. For example in the offline rendering world there's Nvidia's OptiX denoiser and Intel's OIDN, and in the realtime world there's Nvidia's DLSS Ray Reconstruction which uses an ML model to do both upscaling and denoising at the same time.

https://developer.nvidia.com/optix-denoiser

https://www.openimagedenoise.org

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#33

Post by holoduke »

crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

Its still hard to do realtime. You need so much gpu memory that a second GPU must be used at least today. The question is what gets achieved quicker. Hard calculated simulation or AI post processing. Or maybe a combination?

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#34

Post by holoduke »

lispisok wrote: post #12
crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

This is an interesting idea but please no more AI graphics generation in video games please. Games dont get optimized anymore because devs rely on AI upscaling and frame generation to get playable framerates and it makes the games look bad and play bad.

No its because hardware is not fast enough. Performance optimization is a large part of engine development. It happens at Epic as well.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#35

Post by firtoz »

lisyarus wrote: post #26
artemonster wrote: post #23

> "GPU "software" raytracer" > WebGPU > this project is desktop-only Boss, I am confused, boss.

I'm using WebGPU as a nice modern graphics API that is at the same time much more user-friendly and easier to use compared to e.g. Vulkan. I'm using a desktop implementation of WebGPU called wgpu, via it's C bindings called wgpu-native. My browser doesn't support WebGPU properly yet, so I don't really care about running this thing in browser.

That's a fascinating approach.

And it gets me a bit sad about the state of WebGPU, however hopefully that'll be resolved soon... I also on Linux am impatiently waiting for WebGPU to be supported on my browser.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#37

Post by sspiff »

pjmlp wrote: post #22

WebGPU projects that don't provide browser examples are kind of strange, then better use Vulkan or whatever.

WegGPU is a way nicer HAL if you're not an experienced graphics engineer. So even if you only target desktops, it's a valid choice.

On the web, WebGPU is only supported by Chrome-based browser engines at this point, and a lot of software developers us Firefox (and don't really like encouraging a browser monoculture), so it doesn't make a ton of sense to target browser based WebGPU for some people at this point.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#39

Post by lisyarus »

sspiff wrote: post #37
pjmlp wrote: post #22

WebGPU projects that don't provide browser examples are kind of strange, then better use Vulkan or whatever.

WegGPU is a way nicer HAL if you're not an experienced graphics engineer. So even if you only target desktops, it's a valid choice. On the web, WebGPU is only supported by Chrome-based browser engines at this point, and a lot of software developers us Firefox (and don't really like encouraging a browser monoculture), so it doesn't make a ton of sense to target browser based WebGPU for some people at this point.

It's not as much about experience as it is about trade-offs. I've worked a lot with Vulkan and it's an incredible API, but when you're working alone and you don't have the goal of squeezing 250% performance out of your GPU on dozens of different GPU architectures, your performance becomes pretty much independent of a specific graphics API (unless your API doesn't support some stuff like multi-draw-indirect, etc).

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#40

Post by pjmlp »

sspiff wrote: post #37
pjmlp wrote: post #22

WebGPU projects that don't provide browser examples are kind of strange, then better use Vulkan or whatever.

WegGPU is a way nicer HAL if you're not an experienced graphics engineer. So even if you only target desktops, it's a valid choice. On the web, WebGPU is only supported by Chrome-based browser engines at this point, and a lot of software developers us Firefox (and don't really like encouraging a browser monoculture), so it doesn't make a ton of sense to target browser based WebGPU for some people at this point.

The answer is middleware engine, all of them with much nicer tooling available, without the constraints of a browser sandboxing design, for 2017 graphics APIs minimum common denominator.
Post reply on HN