Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

github.com

Post reply on HN
Print view View original

11–20 of 43 posts

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#11

Post by omolobo »

crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

See: https://research.nvidia.com/labs/rtr/tag/neural-rendering/

Specifically this one, which seems to tackle what you mentioned: https://research.nvidia.com/labs/rtr/publication/hadadan2023...

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#12

Post by lispisok »

crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

This is an interesting idea but please no more AI graphics generation in video games please. Games dont get optimized anymore because devs rely on AI upscaling and frame generation to get playable framerates and it makes the games look bad and play bad.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#13

Post by omolobo »

lisyarus wrote: post #7
omolobo wrote: post #5

It's a mega-kernel, so you'll get poor occupancy past the first bounce. A better strategy is to shoot, sort, and repeat, which then also allows you to squeeze in an adaptive sampler in the middle. > // No idea where negative values come from :( I don't know, but: > newRay.origin += sign(dot(newRay.direction, geometryNormal)) * geometryNormal * 1e-4; The new origin should be along the reflected ray, not along the dire…

> It's a mega-kernel, so you'll get poor occupancy past the first bounce Sure! If you look into the to-do list, there's a "wavefront path tracer" entry :) > new origin should be along the reflected ray I've found that doing it the way I'm doing it works better for preventing self-intersections. Might be worth investigating, though.

It probably works better when the reflected ray is almost tangent to the surface. But that should be an epsilon case.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#14

Post by toshinoriyagi »

Etheryte wrote: post #8
crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

At any reasonable quality, AI is even more expensive than raytracing. A simple intuition for this is the fact that you can easily run a raytracer on consumer hardware, even if at low FPS, meanwhile you need a beefy setup to run most AI models and they still take a while.

While some very large models may need beefy hardware, there are multiple forms of deep learning used for similar purposes:

Nvidia's DLSS is a neural network that upscales images so that games may be rendered quickly at lower resolutions, and than upscaled to the display resolution in less total time than rendering natively at the display resolution.

Nvidia's DLDSR downscales a greater-than-native resolution image faster than typical downscaling algorithms used in DSR.

Nvidia's RTX HDR is a post-processing filter that takes an sRGB image and converts it to HDR.

So, it is very likely that a model that converts rasterized images to raytraced versions is possible, and fast. The most likely road block is the lack of a quality dataset for training such a model. Not all games have ray tracing, and even fewer have quality implementations.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#15

Post by TuringTest »

Etheryte wrote: post #8
crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

At any reasonable quality, AI is even more expensive than raytracing. A simple intuition for this is the fact that you can easily run a raytracer on consumer hardware, even if at low FPS, meanwhile you need a beefy setup to run most AI models and they still take a while.

Yeah but that has something to do with

1) commercial hardware pipelinea being improved for decades in handling 3D polygons, and

2) graphical AI models are trained on understanding natural language in addition to rendering.

I can imagine a new breed of specialized generative graphical AI that entirely skips language and is trained on stock 3D objects as input, which could potentially perform much better.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#16

Post by 8n4vidtmkvmk »

Etheryte wrote: post #8
crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

At any reasonable quality, AI is even more expensive than raytracing. A simple intuition for this is the fact that you can easily run a raytracer on consumer hardware, even if at low FPS, meanwhile you need a beefy setup to run most AI models and they still take a while.

I'm not convinced. We have "hyper" and "lightning" diffusion models that run 1-4 steps and are pretty quick on consumer hardware. I really have no idea which would be quicker with some optimizations and hardware tailored for the use-case.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#17

Post by jsheard »

8n4vidtmkvmk wrote: post #16
Etheryte wrote: post #8

Earlier quoted context omitted.

At any reasonable quality, AI is even more expensive than raytracing. A simple intuition for this is the fact that you can easily run a raytracer on consumer hardware, even if at low FPS, meanwhile you need a beefy setup to run most AI models and they still take a while.

I'm not convinced. We have "hyper" and "lightning" diffusion models that run 1-4 steps and are pretty quick on consumer hardware. I really have no idea which would be quicker with some optimizations and hardware tailored for the use-case.

The hard part is keeping everything coherent over time in a dynamic scene with a dynamic camera. Hallucinating vaguely plausible lighting may be adequate for a still image, but not so much in a game if you hallucinate shadows or reflections of off-screen objects that aren't really there, or "forget" that off-screen objects exist, or invent light sources that make no sense in context.

The main benefit of raytracing in games is that it has accurate global knowledge of the scene beyond what's directly in front of the camera, as opposed to earlier approximations which tried to work with only what the camera sees. Img2img diffusion is the ultimate form of the latter approach in that it tries to infer everything from what the camera sees, and guesses the rest.

Post reply on HN