Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Triangle splatting: radiance fields represented by triangles

trianglesplatting.github.io

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Triangle splatting: radiance fields represented by triangles

#61

Post by haeric »

sorenjan wrote: post #27

This looks really nice, but I cant help to think this is a stop gap solution like other splatting techniques. It's certainly better than NERFs, where the whole scene is contained in a black box, but reality is not made up of a triangle soup or gaussian blobs. Most of the real world is made up of volumes, but can often be thought of as surfaces. It makes sense to represent the ground, a table, walls, etc with planes,…

> I haven't done any graphics programming in years, but I thought you'd want to keep the number of draw calls down, do you need to cluster these triangles into fewer draw calls?

GPUs draw can draw 10,000's of vertices per draw call, whether they are connected together into logical objects or are "triangle soup" like this. There is some benefit to having triangles connected together so they can "share" a vertex, but not as much as you might think. Since GPUs are massively parallel, it does not matter much where on the screen or where in the buffer your data is.

> Is there any work being done to optimize a volumetric representation of scenes and from that create a set of surfaces with realistic looking shaders or similar?

This is basically where the field was going until nerfs and splats. But then nerfs and splats were such HUGE steps in fidelity, it inspired a ton of new research towards it, and I think rightfully so! Truth is that reality is really messy, so trying to reconstruct logically separated meshes for everything you see is a very hard way to try to recreate reality. Nerfs and splats recreate reality much easier.

Re: Triangle splatting: radiance fields represented by triangles

#62

Post by spencerflem »

sorenjan wrote: post #52
hirako2000 wrote: post #50

Earlier quoted context omitted.

A digital image is a soup, of RGB dots of various size. Gaussian Splatting radically changed the approach to photogrammetry. Prior approaches to generate surface models, and mapping the captures to materials that a renderer would more or less rasterize with physically accuracy were hitting the ceiling of the technique. NerF was also a revolution but is very compute intensive. Even a browser, a mid range GPU, can rend…

No, an image is a well ordered grid of pixels. The 3D variant would be voxels, and Nvidia recently released a project to do scene reconstruction with sparse voxels [0]. If you take these triangles, make them share vertices, and order them in a certain way, you have a mesh. You can then combine some of them into larger flat surfaces when that makes sense, draw thousands of them in one draw call, calculate intersection…

I still love this older paper on Plenoxels : https://alexyu.net/plenoxels/

It made so much sense to me: voxels with view dependent color, using eg. spherical gaussians.

I don't know how it compares to newer techniques, probably badly since nobody seems to be talking about it.

Re: Triangle splatting: radiance fields represented by triangles

#63

Post by robotresearcher »

growlNark wrote: post #26
enriquto wrote: post #18

Earlier quoted context omitted.

it's the same list of authors, written in different formats the first one is SURNAME, NAME separated by "and" the second one is NAME SURNAME separated by commas The second one is easier to read by humans, but the first one makes it clearer what is the surname (which would be ambiguous otherwise, when there are composite names). But then again, the first format breaks when someone has "and" in their name, which is not…

Why do they use "and"? Why not use an unambiguous joining token like `/`? This just feels like an abuse of informal language to produce fundamentally formal data. As it stands, it certainly does not resemble readable or parseable english.

It’s BibTex format. It’s ancient, ubiquitous, very fussy, and reads badly for humans in some cases. But it’s what we’ve been using since the 1980s.

‘Better’ formats have been proposed but none have stuck nearly as well. It works, and there’s tooling for it.

Re: Triangle splatting: radiance fields represented by triangles

#64

Post by marcellus23 »

jtolmar wrote: post #60
jplusequalt wrote: post #54

Can anyone detail the use case for gaussian splatting to me? What are we trying to solve, or, where direction are we trying to head towards? I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.

Gaussian splatting models the scene as a bunch of normal distributions (fuzzy squished spheres) instead of triangles, then renders those with billboarded triangles. It has advantages (simpler representation, easy to automatically capture from a scan) and disadvantages (not what the hardware is designed for, not watertight). The biggest disadvantage is that most graphics techniques need to be reinvented for it, and it…

what does "not watertight" mean?

Re: Triangle splatting: radiance fields represented by triangles

#65

Post by lambdaone »

marcellus23 wrote: post #64
jtolmar wrote: post #60

Earlier quoted context omitted.

Gaussian splatting models the scene as a bunch of normal distributions (fuzzy squished spheres) instead of triangles, then renders those with billboarded triangles. It has advantages (simpler representation, easy to automatically capture from a scan) and disadvantages (not what the hardware is designed for, not watertight). The biggest disadvantage is that most graphics techniques need to be reinvented for it, and it…

what does "not watertight" mean?

They don't create contiguous surfaces, and GPUs are optimized to deal with sets of triangles that share vertices (a vertex typically being shared by four to six triangles), rather than not shared at all as with this.

"Watertight" is a actially a stronger criterion, which requires not only a contigous surface, but one which encloses a volume without any gaps, but "not watertight" suffices for this.

Re: Triangle splatting: radiance fields represented by triangles

#66

Post by jameshart »

growlNark wrote: post #32
bowsamic wrote: post #28

Earlier quoted context omitted.

It’s how the bibtex author field is defined. You don’t get free choice here. As far as I’m aware bibtex defines and as the separator https://bibtex.eu/fields/author/

Yea but like... why? Typically you use human language operators to produce readable phrases, and this doesn't even approach readable english.

The person who designed it was solving primarily for lexical sorting of the author field, thought maybe having more than two authors was an edge case, and wanted the two author case to be a logical extension of the single author one?

Re: Triangle splatting: radiance fields represented by triangles

#67

Post by vessenes »

taylorius wrote: post #38
Daub wrote: post #35

Earlier quoted context omitted.

Practically, what differentiateS a splat from standard photogrammetry is that it can capture things like reflections, transparency and skies. A standard photogram of (for example) a mirror would confuse the reflection in the mirror for a space behind the mirror. A photogram of a sheet of glass would likewise suffer. The problem is that any tool or process that converts splats into regular geometry produces plain old…

I've never been quite clear on how Splats encode specular (directional) effects. Are they made to only be visible from a narrow field of view (so you see a different splat for different view angles?) or do they encode the specular stuff internally somehow?

Because you have source imagery and colors (and therefore specular and reflective details) from different angles you can add a view angle and location based component to the material/color function; so the material is not just f(point in 3d space) it’s f(pt, view loc, view direction). That’s made differentiable and so you get viewpoint dependent colors for ‘free’.

Re: Triangle splatting: radiance fields represented by triangles

#68

Post by marcellus23 »

lambdaone wrote: post #65
marcellus23 wrote: post #64

Earlier quoted context omitted.

what does "not watertight" mean?

They don't create contiguous surfaces, and GPUs are optimized to deal with sets of triangles that share vertices (a vertex typically being shared by four to six triangles), rather than not shared at all as with this. "Watertight" is a actially a stronger criterion, which requires not only a contigous surface, but one which encloses a volume without any gaps, but "not watertight" suffices for this.

Interesting thank you!

Re: Triangle splatting: radiance fields represented by triangles

#69

Post by walterlw »

machiaweliczny wrote: post #56
jplusequalt wrote: post #54

Can anyone detail the use case for gaussian splatting to me? What are we trying to solve, or, where direction are we trying to head towards? I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.

AFAIK Gaussian Splatting is somehow connected to NeRFs (neural radiance fields), so job of turning multiple 2D images into 3D scene. Actually tried doing something like this recently for drone navigation (using older point cloud methods) but no luck so far. Can anyone who read this suggest something to use to scan room geometry using camera only in real-time (with access to beefy NVIDIA computer if needed) for drone…

have you tried ORB SLAM v3?

Re: Triangle splatting: radiance fields represented by triangles

#70

Post by sorenjan »

spencerflem wrote: post #62
sorenjan wrote: post #52

Earlier quoted context omitted.

No, an image is a well ordered grid of pixels. The 3D variant would be voxels, and Nvidia recently released a project to do scene reconstruction with sparse voxels [0]. If you take these triangles, make them share vertices, and order them in a certain way, you have a mesh. You can then combine some of them into larger flat surfaces when that makes sense, draw thousands of them in one draw call, calculate intersection…

I still love this older paper on Plenoxels : https://alexyu.net/plenoxels/ It made so much sense to me: voxels with view dependent color, using eg. spherical gaussians. I don't know how it compares to newer techniques, probably badly since nobody seems to be talking about it.

They're mentioned in the SVRaster paper.

https://svraster.github.io/images/teaser.jpg

Post reply on HN