31–40 of 59 posts
Looks ok, albeit blurry. Would have been nice to see comparison of render-time between the neural and classical renderers.
The animations (specifically Animated Crab and Robot Animation) have quite noticeable AI art artifacts that swirl around the model in unnatural ways as the objects and camera move.
This is a stellar and interesting idea: train a transformer to turn a scene description set of triangles into a 2d array of pixels, which happens to look like the pixels a global illumination renderer would output from the same scene. That this works at all shouldn’t be shocking after the last five years of research, but I still find it pretty profound. That transformer architecture sure is versatile. Anyway, crazy f…
The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This could possibly enable higher quality instant render previews for 3D designers in web or native apps using on-dev…
I think they’ve inadvertently included Blender’s instantiation phase in the overall rendering time, while not including the transformer instantiation.
I’d be interested to see the time to render the second frame for each system. My hunch is that Blender would be a lot more performant.
I do think the papers results are fascinating in general, but there’s some nuance in the way they’ve configured and timed Blender.
HN what do you think are interesting non-text domains where transformers would be well suited?
Earlier quoted context omitted.
For the scenes that they’re showing, 76ms is an eternity. Granted, it will get (a lot) faster but this being better than traditional rendering is a way off yet.
Yeah, and the big caveat with this approach is that it scales quadratically with scene complexity, as opposed to the usual methods which are logarithmic. Their examples only have 4096 triangles at most for that reason. It's a cool potential direction for future research but there's a long way to go before it can wrangle real production scenes with hundreds of millions of triangles.
I don't think this conventional similarity matrix in the paper is all that important to them
Earlier quoted context omitted.
I've had a chat with AI about this exact thing, using this paper as input. I started with the idea of using a light probe/triangle, but things got ugly fast. Below is the final message from the AI: *You're absolutely right!* This is exactly why RenderFormer's approach is actually brilliant. *The complexity spiral you just discovered:* - Start with "simple" differential spherical harmonics - Add spatial data structure…
I'm sorry, but I really don't think posting AI chat logs one has had about the given topic is a meaningful or constructive input to threads like this. Concievably, you could have had the chat session and--assuming the exercise gave you new insights--replied as yourself with those insights. But this, just posting the log, is both difficult to read and feels like you didn't put much effort into replying to the conversa…
My apologies.
Earlier quoted context omitted.
> The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This sounds pretty wild to me. Scanned through it quickly but I couldn't find any details on how they set this up.…
This reads like they used the GPU with Cycles: "Table 2 compares the timings on the four scenes in Figure 1 of our unoptimized RenderFormer (pure PyTorch implementation without DNN compilation, but with pre-caching of kernels) and Blender Cy- cles with 4,096 samples per pixel (matching RenderFormer’s training data) at 512 × 512 resolution on a single NVIDIA A100 GPU."
This seems like an unfair comparison. It would be a lot more useful to know how long it would take Blender to also reach a 0.9526 Structural Similarity Index Measure to the training data. My guess is that with the de-noiser turned on, something like 128 samples would be enough, or maybe even less on some images. At that point on an A100 GPU Blender would be close, if not beating the times here for these scenes.
Earlier quoted context omitted.
> The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This sounds pretty wild to me. Scanned through it quickly but I couldn't find any details on how they set this up.…
This reads like they used the GPU with Cycles: "Table 2 compares the timings on the four scenes in Figure 1 of our unoptimized RenderFormer (pure PyTorch implementation without DNN compilation, but with pre-caching of kernels) and Blender Cy- cles with 4,096 samples per pixel (matching RenderFormer’s training data) at 512 × 512 resolution on a single NVIDIA A100 GPU."
The coolest thing here might be the speed: for a given scene RenderFormer takes 0.0760 seconds while Blender Cycles takes 3.97 seconds (or 12.05 secs at a higher setting), while retaining a 0.9526 Structural Similarity Index Measure (0-1 where 1 is an identical image). See tables 2 and 1 in the paper. This could possibly enable higher quality instant render previews for 3D designers in web or native apps using on-dev…
I don’t think the authors are being wilfully deceptive in any way, but Blender Cycles on a gpu of that quality could absolutely render every scene in this paper in less than 4s per frame. There are very modest tech demo scenes with low complexity, and they’ve set blender to cycle through 4k iterations per pixel - which seems non-sensible as Blender would hit something close to its output after a couple of hundred cyc…
Blenders benchmark database doesn't have any results for the A100, but even the newer H100 gets smoked by (relatively) cheap consumer hardware.
Nvidia H100 NVL - 5,597.13
GeForce RTX 3090 Ti - 5,604.69
Apple M3 Ultra (80C) - 7,319.21
GeForce RTX 4090 - 11,082.51
GeForce RTX 5090 - 15,022.02
RTX PRO 6000 Blackwell - 16,336.54