Earlier quoted context omitted.
And in case it helps further in the context of the article: traditional rendering pipelines for games don't render fuzzy Gaussian points, but triangles instead. Having the model trained on how to construct triangles (rather than blobbly points) means that we're closer to a "take photos of a scene, process them automatically, and walk around them in a game engine" style pipeline.
Any insights into why game engines prefer triangles rather than guassians for fast rendering? Are triangles cheaper for the rasterizer, antialiasing, or something similar?
Triangle splatting: radiance fields represented by triangles
51–60 of 81 posts
Re: Triangle splatting: radiance fields represented by triangles
#52This looks really nice, but I cant help to think this is a stop gap solution like other splatting techniques. It's certainly better than NERFs, where the whole scene is contained in a black box, but reality is not made up of a triangle soup or gaussian blobs. Most of the real world is made up of volumes, but can often be thought of as surfaces. It makes sense to represent the ground, a table, walls, etc with planes,…
A digital image is a soup, of RGB dots of various size. Gaussian Splatting radically changed the approach to photogrammetry. Prior approaches to generate surface models, and mapping the captures to materials that a renderer would more or less rasterize with physically accuracy were hitting the ceiling of the technique. NerF was also a revolution but is very compute intensive. Even a browser, a mid range GPU, can rend…
If you take these triangles, make them share vertices, and order them in a certain way, you have a mesh. You can then combine some of them into larger flat surfaces when that makes sense, draw thousands of them in one draw call, calculate intersections, volumes, physics, LODs, use textures with image compression instead of millions of colored objects, etc with them. Splatting is one way of answering the question "how do we reproduce these images in a way that lets us generate novel views of the same scene", not "what is the best representation of this 3D scene".
The aim is to find the light field that describes the scene, and if you have solid objects that function can be described on the surface of those objects. Seems like a much more elegant end result than a cloud of separate objects, no matter what shape they have, since that's much closer to how reality works. Obviously we need to handle volumetrics and translucency as well, but if we model the real surfaces as virtual surfaces I think things like reflections and shadow removal will be easier. At least gaussian splats have a hard time with reflections, they look good from some viewing angles, but the reflections are often handled as geometry [1].
I'm not arguing that it doesn't look good or that it doesn't serve a purpose, sometimes a photorealistic novel view of a real scene is all you want. But I still don't think it's the best representation of scenes.
Re: Triangle splatting: radiance fields represented by triangles
#53Earlier quoted context omitted.
And in case it helps further in the context of the article: traditional rendering pipelines for games don't render fuzzy Gaussian points, but triangles instead. Having the model trained on how to construct triangles (rather than blobbly points) means that we're closer to a "take photos of a scene, process them automatically, and walk around them in a game engine" style pipeline.
Any insights into why game engines prefer triangles rather than guassians for fast rendering? Are triangles cheaper for the rasterizer, antialiasing, or something similar?
Yes, using triangles simplifies a lot of math, and GPUs were created to be really good at doing the math related to triangles rasterization (affine transformations).
Re: Triangle splatting: radiance fields represented by triangles
#54I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.
Re: Triangle splatting: radiance fields represented by triangles
#55Can anyone detail the use case for gaussian splatting to me? What are we trying to solve, or, where direction are we trying to head towards? I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.
Re: Triangle splatting: radiance fields represented by triangles
#56Can anyone detail the use case for gaussian splatting to me? What are we trying to solve, or, where direction are we trying to head towards? I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.
Can anyone who read this suggest something to use to scan room geometry using camera only in real-time (with access to beefy NVIDIA computer if needed) for drone navigation purposes?
Re: Triangle splatting: radiance fields represented by triangles
#57Re: Triangle splatting: radiance fields represented by triangles
#58Re: Triangle splatting: radiance fields represented by triangles
#59Can anyone detail the use case for gaussian splatting to me? What are we trying to solve, or, where direction are we trying to head towards? I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.
I get the impression the goal is to save 3D environments with baked lighting without having to run raytracing, at a level above explicitly defined meshes with faces covered by 2D textures, which can't represent fog, translucency, reflection glints, etc without a separate lighting pass. Basically trying to get raytracing without doing raytracing.
Re: Triangle splatting: radiance fields represented by triangles
#60Can anyone detail the use case for gaussian splatting to me? What are we trying to solve, or, where direction are we trying to head towards? I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.