I’m impressed by Gaussian Splatting and I love these kind of demos, but I have a noob question - can they respond to lighting?
Show HN: Gaussian Splat of a Strawberry
171–180 of 211 posts
Re: Show HN: Gaussian Splat of a Strawberry
#172This is like a beautiful little miniature. It's cool to see gaussian splatting applied to a detailed small thing versus a big scene. I have a question about perhaps the most boring aspect of this strawberry: the license. you write, "You can download it under CC BY license, but attribution is appreciated rather than required." IANAL, but I'm pretty sure you can't license your work CC-BY and then waive the BY requireme…
It's their content. They can do anything they like.
Sort of. There are countries like France where attribution rights are fundamentally inalienable, and the author can always demand attribution, even after forever waiving the right to attribution. But in the US, the party who benefits from a contractual provision can generally choose to ignore or forfeit that provision. Whether it's still called the "CC BY" after that is debatable, since the whole point is attribution, but licensing by the CC BY and immediately waiving that requirement seems legitimate to me.
Re: Show HN: Gaussian Splat of a Strawberry
#173Earlier quoted context omitted.
There are many ways to represent 3D data, but animations really only work properly with polygon meshes (e.g. triangle surface meshes or volumetric tetrahedral meshes).
But the mesh is itself an abstraction, you just need to build that bridge. We've been leaning away from pure polygons for decades, anyway. Vertex skinning, SDFs, volumetrics, simulation, and a lot more. The meshes in a From Software game are for exmple hilariously simple, most of the animation is force simulation to make the famous "frizzles" that they like.
Vertex skinning is essential for animation and it only works with polygons.
Re: Show HN: Gaussian Splat of a Strawberry
#174I’m impressed by Gaussian Splatting and I love these kind of demos, but I have a noob question - can they respond to lighting?
Not traditional Gaussian splats since they have their lighting baked in but there are alternate forms that have the ability to do so but largely in academia or very specific use cases
Re: Show HN: Gaussian Splat of a Strawberry
#175Earlier quoted context omitted.
It appears to be rendering locally, my fans started spinning up when I was wandering through another image on the site.
Where else would it render??
I just meant that it's rendering the 3D view live in your browser, not streaming a video of the rendered output
Re: Show HN: Gaussian Splat of a Strawberry
#176Also interesting: https://github.com/apple/ml-sharp Apples model to generate Gaussian splats from a single image. Takes about 30 seconds on an M1 Pro. It falls apart once you move too much, but for a little side-wiggling or a second-eye view for VR, it's great. And looks a lot better than the old approach of depth map + vertex shaders that I use in https://github.com/combatwombat/tiefling . But ml-sharp has 2.6 GB we…
Is this thing the basis of what Apple is doing in the Photos app in the most recent OS revisions?
It seems to work a lot quicker than 30s now on iDevices and Macs.
Re: Show HN: Gaussian Splat of a Strawberry
#177I read [1], but I still don't quite know what I'm looking at. My guess is a 3D model reconstructed from lots of detailed pictures? [1] https://en.wikipedia.org/wiki/Gaussian_splatting
Here’s a good 2 minute explainer https://youtu.be/HVv_IQKlafQ
Oh thanks - I was waiting for a moment where I could turn up sound to watch the other video, but I didn't realise that that would set me back half an hour. This is the perfect amount of background for now!
Re: Show HN: Gaussian Splat of a Strawberry
#178Earlier quoted context omitted.
But the mesh is itself an abstraction, you just need to build that bridge. We've been leaning away from pure polygons for decades, anyway. Vertex skinning, SDFs, volumetrics, simulation, and a lot more. The meshes in a From Software game are for exmple hilariously simple, most of the animation is force simulation to make the famous "frizzles" that they like.
Vertex skinning is essential for animation and it only works with polygons.
I'm not sure that's completely accurate? Vertex skinning isn't (necessarily) tied to polygons . . but to having points (or any parameterized features) that can be transformed by a weighted blend of matrices.
The "vertex" in "vertex skinning" is really just "a thing with a position that gets moved."
p' = Σ wᵢ · Mᵢ · p
It's just a position. Triangles can come along for the ride downstream, but they're not essential, which is one of the reasons it's so efficient for some stuff. Polygons are the optimal surface - but surfaces are often extraneous.
Take all this a few hefty grains of salt, I'm an amateur in the field. My 3d/CAD work is strictly in support of my enterprise stuff. And making wicked battlemaps for gaming VTTs, natch.
But I will stand by the overarching statement that polygons are in fact an abstraction, and bridging that abstraction with whatever is in splats would be wicked awesome.
Re: Show HN: Gaussian Splat of a Strawberry
#179This took me down a rabbit hole that led to this company, doing Gaussian splat videos: https://www.4dv.ai/. Fascinating.
Re: Show HN: Gaussian Splat of a Strawberry
#180I read [1], but I still don't quite know what I'm looking at. My guess is a 3D model reconstructed from lots of detailed pictures? [1] https://en.wikipedia.org/wiki/Gaussian_splatting
Others have provided details about how it works. I suggest zooming way in on that image and you'll start 'breaking though' the surface and that'll help you get an idea of how it works. Important thing is there is no defined geometric surface ("mesh"). Also important to know is that it's very, very hard to get a good splat without taking a ton of photos at different angles. It's also really, really easy to create a cr…
I don't know, there's plenty of models these days that generate good splats just from objects at home.
I took maybe 10 pictures of a model I built and threw it at my 3060 during dinner and it came out quite nice.