Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Gaussian Splat of a Strawberry

superspl.at

Post reply on HN
Print view View original

111–120 of 211 posts

Re: Show HN: Gaussian Splat of a Strawberry

#111

Post by sovok »

RobotToaster wrote: post #107
sovok wrote: post #105

Also interesting: https://github.com/apple/ml-sharp Apples model to generate Gaussian splats from a single image. Takes about 30 seconds on an M1 Pro. It falls apart once you move too much, but for a little side-wiggling or a second-eye view for VR, it's great. And looks a lot better than the old approach of depth map + vertex shaders that I use in https://github.com/combatwombat/tiefling . But ml-sharp has 2.6 GB we…

Any idea what kind of vram is needed to run this on Nvidia?

The Python process peaks at about 15 GB, so maybe a 24 GB GPU works. But others seem to have problems wih Nvidia: https://github.com/apple/ml-sharp/issues/84

Re: Show HN: Gaussian Splat of a Strawberry

#112

Post by lightedman »

A Youtuber that records jookin (a mid-South form of dance) uses Gaussian splats for his videos to bring the dancers out into the forefront and hold center stage, it's a neat trick.

https://www.youtube.com/watch?v=k0MGoFaIiWs - WARNING - heavily adult lyrics and a loud starting "MOTHERFU*AAAAAAS" right off the top.

Re: Show HN: Gaussian Splat of a Strawberry

#113

Post by stevepotter »

Vinnl wrote: post #37

I read [1], but I still don't quite know what I'm looking at. My guess is a 3D model reconstructed from lots of detailed pictures? [1] https://en.wikipedia.org/wiki/Gaussian_splatting

Others have provided details about how it works. I suggest zooming way in on that image and you'll start 'breaking though' the surface and that'll help you get an idea of how it works. Important thing is there is no defined geometric surface ("mesh"). Also important to know is that it's very, very hard to get a good splat without taking a ton of photos at different angles. It's also really, really easy to create a crappy looking splat. But when it's done right, it's a marvel

Re: Show HN: Gaussian Splat of a Strawberry

#114

Post by cubefox »

perching_aix wrote: post #92
MattCruikshank wrote: post #88

I know feelings about AI are mixed. But when AI can dream up gaussian splats in real time, from a prompt, and do refinement as you get closer to things... That's going to be pretty bonkers.

That's kinda what NERFs are (neural radience fields). They actually preceeded this Gaussian story, with Gaussians coming in and outperforming them. Maybe they'll merge later for something even better, I don't know enough about them.

NERFs have significantly higher image quality than 3D Gaussian Splatting or more recent similar techniques, though they are much slower to render.

Re: Show HN: Gaussian Splat of a Strawberry

#115

Post by divan »

Does anyone know if it's already possible to build gaussian splat of the person that moves/rotates from the single camera? (I.e. to use sequence of frames to reconstruct occluded parts of the body for other frames)

Re: Show HN: Gaussian Splat of a Strawberry

#116

Post by jl6 »

xattt wrote: post #82
Tade0 wrote: post #39

Beautiful. What I love about gaussian splats is the way they degrade - instead of a hard cutoff or LoD changing spheres into cubes etc., they get increasingly "dreamy" - the basic idea is still there, just less detailed. Take for example this scene: https://superspl.at/scene/e721ea7c If you navigate closer to the trees, things around you become blurry - as if the very fabric of reality unraveled.

Is there a specific name for this “dreaminess” effect?

Splattenuation

Re: Show HN: Gaussian Splat of a Strawberry

#118

Post by kridsdale1 »

RobotToaster wrote: post #107
sovok wrote: post #105

Also interesting: https://github.com/apple/ml-sharp Apples model to generate Gaussian splats from a single image. Takes about 30 seconds on an M1 Pro. It falls apart once you move too much, but for a little side-wiggling or a second-eye view for VR, it's great. And looks a lot better than the old approach of depth map + vertex shaders that I use in https://github.com/combatwombat/tiefling . But ml-sharp has 2.6 GB we…

Any idea what kind of vram is needed to run this on Nvidia?

It runs natively on the Photos app in Vision Pro, which shares 16GB of ram with GPU and CPU.

Re: Show HN: Gaussian Splat of a Strawberry

#119

Post by kridsdale1 »

jcattle wrote: post #25
josh-wrale wrote: post #4

Someone: Please combine microscopy with gaussian splatting.

Don't know if this would be in your wheelhouse, but for very nice macro splats, check out the work by Dany Bittel: https://danybittel.ch/macro.html For example this bumblebee: https://superspl.at/scene/cf6ac78e Edit: I completely missed that this was posted by him (:

Incredible that specular highlight isotropy vectors are somehow encoded.

Re: Show HN: Gaussian Splat of a Strawberry

#120

Post by MattCruikshank »

Lerc wrote: post #104
MattCruikshank wrote: post #88

I know feelings about AI are mixed. But when AI can dream up gaussian splats in real time, from a prompt, and do refinement as you get closer to things... That's going to be pretty bonkers.

This will be the future of a class of 3d Game. the prompt may not be text however. An input of a kind of schematic representation of what the designer wants would be better. It may resemble a storyboard or a collection of organised notes that large projects tend to already use. Fully generative could probably do some cool things, but people will still want to bring their peronal vision to life.

Yeah, when you describe that, I picture Wave Function Collapse to generate a map schematic... And then a text prompt, and some style photos the designers want it to match.
Post reply on HN