Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RenderFormer: Neural rendering of triangle meshes with global illumination

microsoft.github.io

Post reply on HN
Print view View original

41–50 of 59 posts

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#41

Post by leloctai »

timhigins wrote: post #8

The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This could possibly enable higher quality instant render previews for 3D designers in web or native apps using on-dev…

Timing comparison with the reference is very disingenuous.

In raytracing, error scale with the square root of sample count. While it is typical to use very high sample count for the reference, real world sample count for offline renderer is about 1-2 orders of magnitude lower than in this paper.

I call it disingenuous because it is very usual for a graphic paper to include a very high sample count reference image for quality comparison, but nobody ever do timing comparison with it.

Since the result is approximate, a fair comparison would be with other approximate rendering algorithm. Modern realtime path tracer + denoiser can render much more complex scenes on consumer GPU in less than 16ms.

That's "much more complex scenes" part is the crucial part. Using transformer mean quadratic scaling on both number of triangles and number of output pixels. I'm not up to date with the latest ML research, so maybe it is improved now? But I don't think it will ever beat O(log n_triangles) and O(n_pixels) theoretical scaling of a typical path tracer. (Practical scaling wrt pixel count is sub linear due to high coherency of adjacent pixels)

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#42

Post by rcxdude »

OtherShrezzing wrote: post #33
timhigins wrote: post #8

The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This could possibly enable higher quality instant render previews for 3D designers in web or native apps using on-dev…

I don’t think the authors are being wilfully deceptive in any way, but Blender Cycles on a gpu of that quality could absolutely render every scene in this paper in less than 4s per frame. There are very modest tech demo scenes with low complexity, and they’ve set blender to cycle through 4k iterations per pixel - which seems non-sensible as Blender would hit something close to its output after a couple of hundred cyc…

Yeah, you would generally set blender to have some low minimum number of cycles, maybe have some adaptive noise target, and use a denoising model, especially for preview or draft renders.

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#43

Post by [deleted] »

moron4hire wrote: post #17
ToJans wrote: post #15

Earlier quoted context omitted.

I've had a chat with AI about this exact thing, using this paper as input. I started with the idea of using a light probe/triangle, but things got ugly fast. Below is the final message from the AI: *You're absolutely right!* This is exactly why RenderFormer's approach is actually brilliant. *The complexity spiral you just discovered:* - Start with "simple" differential spherical harmonics - Add spatial data structure…

I'm sorry, but I really don't think posting AI chat logs one has had about the given topic is a meaningful or constructive input to threads like this. Concievably, you could have had the chat session and--assuming the exercise gave you new insights--replied as yourself with those insights. But this, just posting the log, is both difficult to read and feels like you didn't put much effort into replying to the conversa…

[deleted]

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#44

Post by alpaca128 »

01HNNWZ0MV43FF wrote: post #12
dclowd9901 wrote: post #6

Forgive my ignorance: are these scenes rendered based on how a scene is expected to be rendered? If so, why would we use this over more direct methods (since I assume this is not faster than direct methods)?

Another comment says this is faster. Global illumination can be very slow with direct methods

As others point out it's a biased comparison. Their compared Blender render ran more than 10x as many cycles as usual, ran on a GPU without raytracing acceleration which could make it slower than consumer models, and potentially it also included the startup time of the renderer.

Considering their AI achieved about 96% accuracy to the reference, it would be more interesting to see how Blender does on fitting hardware and with a matching quality setting. Or maybe even a modern game engine.

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#45

Post by CyberDildonics »

With every graphics paper it's important to think about what you don't see. Here there are barely any polygons, low resolution, no textures, no motion blur, no depth of field and there are some artifacts in the animation.

It's interesting research but to put it in perspective this is using modern GPUs to make images that look like what was being done with 1/1,000,000 the computation 30 years ago.

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#46

Post by [deleted] »

CyberDildonics wrote: post #45

With every graphics paper it's important to think about what you don't see. Here there are barely any polygons, low resolution, no textures, no motion blur, no depth of field and there are some artifacts in the animation. It's interesting research but to put it in perspective this is using modern GPUs to make images that look like what was being done with 1/1,000,000 the computation 30 years ago.

[deleted]

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#47

Post by notnullorvoid »

I found it odd that none of the examples showed anything behind the camera. I'm not sure if that's a limitation of the approach or an oversight in creating examples. What I do know is that when we're talking about reflections and lighting what's behind the camera is pretty important.

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#48

Post by cubefox »

leloctai wrote: post #41
timhigins wrote: post #8

The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This could possibly enable higher quality instant render previews for 3D designers in web or native apps using on-dev…

Timing comparison with the reference is very disingenuous. In raytracing, error scale with the square root of sample count. While it is typical to use very high sample count for the reference, real world sample count for offline renderer is about 1-2 orders of magnitude lower than in this paper. I call it disingenuous because it is very usual for a graphic paper to include a very high sample count reference image for…

Modern optimized path tracers in games (probably not Blender) also use rasterization for primary visibility, which is O(n_triangles), but is somehow even faster than doing pure path tracing. I guess because is reduces the number of samples required to resolve high frequency texture details. Global illumination by itself tends to produce very soft (low frequency) shadows and highlights, so not a lot of samples are required in theory, when the denoiser can avoid artifacts at low sample counts.

But yeah, no way RenderFormer in its current state can compete with modern ray tracing algorithms. Though the machine learning approach to rendering is still in its infancy.

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#49

Post by johnb231 »

ToJans wrote: post #37
moron4hire wrote: post #17

Earlier quoted context omitted.

I'm sorry, but I really don't think posting AI chat logs one has had about the given topic is a meaningful or constructive input to threads like this. Concievably, you could have had the chat session and--assuming the exercise gave you new insights--replied as yourself with those insights. But this, just posting the log, is both difficult to read and feels like you didn't put much effort into replying to the conversa…

The gist of my post was in the first few sentences, I just added it for whoever would like to read it in more detail. My apologies.

The point is not made clear in the first few sentences. Ironically you could have used AI to make the post readable. Copy/paste AI slop.

Re: RenderFormer: Neural rendering of triangle meshes with global illumination

#50

Post by coalteddy »

I have a friend that works on physically based renderers in the film industry and has also done research in the area. Always love hearing stories and explanations about how things get done in this industry.

What companies are hiring such talent at the moment? Have the AI companies also been hiring rendering engineers for creating training environments?

If you are looking to hire an experienced research and industry rendering engineer i am happy to connect you since my friend is not on social media but has been putting out feelers.

Post reply on HN