Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

github.com

Post reply on HN
Print view View original

21–30 of 43 posts

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#21

Post by nuclearsugar »

crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

I think there will certainly be an AI 3D render engine at some point. But currently AI is used in 3D render engines to assist with denoising. https://docs.blender.org/manual/en/2.92/render/layers/denois...

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#24

Post by mywittyname »

toshinoriyagi wrote: post #14
Etheryte wrote: post #8

Earlier quoted context omitted.

At any reasonable quality, AI is even more expensive than raytracing. A simple intuition for this is the fact that you can easily run a raytracer on consumer hardware, even if at low FPS, meanwhile you need a beefy setup to run most AI models and they still take a while.

While some very large models may need beefy hardware, there are multiple forms of deep learning used for similar purposes: Nvidia's DLSS is a neural network that upscales images so that games may be rendered quickly at lower resolutions, and than upscaled to the display resolution in less total time than rendering natively at the display resolution. Nvidia's DLDSR downscales a greater-than-native resolution image fas…

> So, it is very likely that a model that converts rasterized images to raytraced versions is possible, and fast.

How would this even work and not just be a DLSS derivative?

The magic of ray tracing is the ability to render light sources and reflections that are not in the scene. So where is the information coming from that the algorithm would use to place and draw the lights, shadows, reflections, etc?

I'm not asking to be snarky. I can usually "get there from here" when it comes to theoretical technology, but I can't work out how a raster image would contain enough data to allow for accurate ray tracing to be applied for objects whose effects are only included due to ray tracing.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#25

Post by jsheard »

toshinoriyagi wrote: post #14
Etheryte wrote: post #8

Earlier quoted context omitted.

At any reasonable quality, AI is even more expensive than raytracing. A simple intuition for this is the fact that you can easily run a raytracer on consumer hardware, even if at low FPS, meanwhile you need a beefy setup to run most AI models and they still take a while.

While some very large models may need beefy hardware, there are multiple forms of deep learning used for similar purposes: Nvidia's DLSS is a neural network that upscales images so that games may be rendered quickly at lower resolutions, and than upscaled to the display resolution in less total time than rendering natively at the display resolution. Nvidia's DLDSR downscales a greater-than-native resolution image fas…

To be clear DLSS is a very different beast than your typical AI upscaler, it uses the principle of temporal reuse where real samples from previous frames are combined with samples from the current frame in order to converge towards a higher resolution over time. It's not guessing new samples out of thin air, just guessing whether old samples are still usable, which is why DLSS is so fast and accurate compared to general purpose AI upscalers and why you can't use DLSS on images or videos.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#26

Post by lisyarus »

artemonster wrote: post #23

> "GPU "software" raytracer" > WebGPU > this project is desktop-only Boss, I am confused, boss.

I'm using WebGPU as a nice modern graphics API that is at the same time much more user-friendly and easier to use compared to e.g. Vulkan. I'm using a desktop implementation of WebGPU called wgpu, via it's C bindings called wgpu-native.

My browser doesn't support WebGPU properly yet, so I don't really care about running this thing in browser.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#28

Post by TomClabault »

omolobo wrote: post #5

It's a mega-kernel, so you'll get poor occupancy past the first bounce. A better strategy is to shoot, sort, and repeat, which then also allows you to squeeze in an adaptive sampler in the middle. > // No idea where negative values come from :( I don't know, but: > newRay.origin += sign(dot(newRay.direction, geometryNormal)) * geometryNormal * 1e-4; The new origin should be along the reflected ray, not along the dire…

> A better strategy is to shoot, sort, and repeat

Do we have good sorting strategy whose costs are amortized yet? Meister 2020 (https://meistdan.github.io/publications/raysorting/paper.pdf) shows that the hard part is actually to hide the cost of the sorting.

> squeeze in an adaptive sampler in the middle. Can you expand on that? How does that work? I only know of adaptive sampling in screen space where you shoot more or less rays to certain pixels based on their estimated variance so far.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#29

Post by jms55 »

jsheard wrote: post #25
toshinoriyagi wrote: post #14

Earlier quoted context omitted.

While some very large models may need beefy hardware, there are multiple forms of deep learning used for similar purposes: Nvidia's DLSS is a neural network that upscales images so that games may be rendered quickly at lower resolutions, and than upscaled to the display resolution in less total time than rendering natively at the display resolution. Nvidia's DLDSR downscales a greater-than-native resolution image fas…

To be clear DLSS is a very different beast than your typical AI upscaler, it uses the principle of temporal reuse where real samples from previous frames are combined with samples from the current frame in order to converge towards a higher resolution over time. It's not guessing new samples out of thin air, just guessing whether old samples are still usable, which is why DLSS is so fast and accurate compared to gene…

To add to this, DLSS 2 functions exactly the same as a non-ML temporal upscaler does: it blends pixels from the previous frame with pixels from the current frame.

The ML part of DLSS is that the blend weights are determined by a neural net, rather than handwritten heuristics.

DLSS 1 _did_ try and and use neural networks to predict the new (upscaled) pixels outright, which went really poorly for a variety of reasons I don't feel like getting into, hence why they abandoned that approach.

Re: Show HN: I've made a Monte-Carlo raytracer for glTF scenes in WebGPU

#30

Post by jms55 »

crazygringo wrote: post #4

This is a completely side question, but just because it always astonishes me how "real" raytraced scenes can look in terms of lighting, but it's too complex/slow for video games. How far have we gotten in terms of training AI models on raytraced lighting, to simulate it but fast enough for video games? Training an AI not on rendered scenes from any particular viewpoint, but rather on how light and shadows would be "b…

This was a recent presentation from SIGGRAPH 2024 that covered using neural nets to store baked (not dynamic!) lighting https://advances.realtimerendering.com/s2024/#neural_light_g....

Even with the fact that it's static lighting, you can already see a ton of the challenges that they faced. In the end they did get a fairly usable solution that improved on their existing baking tools, but it took what seems like months of experimenting without clear linear progress. They could have just as easily stalled out and been stuck with models that didn't work.

And that was just for static lighting, not every realtime dynamic lighting. ML is going to need a lot of advancements before it can predict lighting whole-sale, faster and easier than tracing rays.

On the other hand ML is really really good at replacing all the mediocre handwritten heuristics 3d rendering has. For lighting, denoising low-signal (0.5-1 rays per pixel) lighting is a big area of research[0] since handwritten heuristics tend to struggle with such little amount of data available, along with lighting caches[1] which have to adapt to a wide variety of situations that again make handwritten heuristics struggle.

[0]: https://gpuopen.com/learn/neural_supersampling_and_denoising..., and the references it lists

[1]:https://research.nvidia.com/publication/2021-06_real-time-ne...

Post reply on HN