[flagged]
RenderFormer: Neural rendering of triangle meshes with global illumination
51–59 of 59 posts
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#52The bitter lesson strikes again… now for graphics rendering. Nerfs had a ray tracing prior, and Gaussian splats had some raster prior. This just… throws it all away. No priors, no domain knowledge, just data and attention.
This is the way.
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#53The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This could possibly enable higher quality instant render previews for 3D designers in web or native apps using on-dev…
I don’t think the authors are being wilfully deceptive in any way, but Blender Cycles on a gpu of that quality could absolutely render every scene in this paper in less than 4s per frame. There are very modest tech demo scenes with low complexity, and they’ve set blender to cycle through 4k iterations per pixel - which seems non-sensible as Blender would hit something close to its output after a couple of hundred cyc…
But rendering engines have been optimized for years and this is a research paper. Probably this technique will also be optimized for years and provide a 10x speedup again
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#54I have a friend that works on physically based renderers in the film industry and has also done research in the area. Always love hearing stories and explanations about how things get done in this industry. What companies are hiring such talent at the moment? Have the AI companies also been hiring rendering engineers for creating training environments? If you are looking to hire an experienced research and industry r…
Have him ping me. Username at Gmail.
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#55[dead]
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#56The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This could possibly enable higher quality instant render previews for 3D designers in web or native apps using on-dev…
Thanks for these comments! Seems their measurement of Blender is off and we need some more in-depth benchmarks.
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#57How efficient is neural rendering at this stage for game rendering?
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#58Deep learning is also very successfully used for denoising of global illumination rendered images [1]. In this approach, traditional raytracing algorithm quickly computes rough global illumination of the scene, and neural network is used to remove noise from the output. . [1] https://www.openimagedenoise.org
The output image of the demo looks uncannily smooth, like an AI upscale. I feel it's what happens when you preserve edges but lose textures when trying to blow up an image past the amount of incoming data it has. (EDIT) Denoising compares better at 100% zoom than 125% DPI zoom, and does make it easier to recognize the ferns at the bottom.
The other thing is that this is supposed to sit in between 3d rendering and compositing with regards to VFX
The 3D render, in an ideal world, is super smooth without imperfections.
The compositing, would take the denoised 3d render, and add other imperfections such as film grain, bloom, and other post effects.
Re: RenderFormer: Neural rendering of triangle meshes with global illumination
#59Earlier quoted context omitted.
I don’t think the authors are being wilfully deceptive in any way, but Blender Cycles on a gpu of that quality could absolutely render every scene in this paper in less than 4s per frame. There are very modest tech demo scenes with low complexity, and they’ve set blender to cycle through 4k iterations per pixel - which seems non-sensible as Blender would hit something close to its output after a couple of hundred cyc…
But rendering engines have been optimized for years and this is a research paper. Probably this technique will also be optimized for years and provide a 10x speedup again
Sure, but algorithmic complexity beats linear factors, so unless they somehow manage to get from O(N²) to O(log N) for triangle count, this technique cannot ever even come close to established traditional approaches; no matter the linear improvement.