Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Turn a single image into a navigable 3D Gaussian Splat with depth

lab.revelium.studio

Post reply on HN
Print view View original

21–30 of 43 posts

Re: Turn a single image into a navigable 3D Gaussian Splat with depth

#21

Post by voodooEntity »

smusamashah wrote: post #18

If this model is so good at estimating depth from single image, shouldn't it also be able to take multiple images as input and estimate even better? But searching a bit it looks like this is supposed to be a single image to 3D only. I don't understand why it does not (can not?) work with multiple images.

If you have multiple images you could use photogrammetry.

At the end, if you want to "fill in the blanks" llm will always "make up" stuff, based on all of its training data.

With a technology like photogrammetry you can get much better results, therefor if you have multiple angled images and dont really need to make up stuff, its better to use such

Re: Turn a single image into a navigable 3D Gaussian Splat with depth

#22

Post by carlosjobim »

Johnny_Bonk wrote: post #8

Cool, is there a way to upload several photos of a room from different angles to fuse it all together? Is there an api?

That is the entire science of photogrammetry. Which has made tremendous progress in the past 10 years. There's many tools which will do it for you.

Re: Turn a single image into a navigable 3D Gaussian Splat with depth

#23

Post by hahahahhaah »

voodooEntity wrote: post #19

Its funny, always stucks on 90% till it fails with the error that another big image may be keeping the server busy. I mean ok its a "demo" tho the funny thing is if you actually check the cli and requests, you clearly can see that the 3 stages the images walks through on "processing" are fake, its just doing 1 post request in the backend that runs while it traverses through the states, and at 90% it stops until (in t…

Oh it's an IE6 progress bar then.

Re: Turn a single image into a navigable 3D Gaussian Splat with depth

#25

Post by echelon »

smusamashah wrote: post #18

If this model is so good at estimating depth from single image, shouldn't it also be able to take multiple images as input and estimate even better? But searching a bit it looks like this is supposed to be a single image to 3D only. I don't understand why it does not (can not?) work with multiple images.

Also, are we allowed to use this model? Apple had a very restrictive licence, IIRC?

Re: Turn a single image into a navigable 3D Gaussian Splat with depth

#26

Post by vunderba »

personjerry wrote: post #13

This is just Apple's tool plus a splat viewing library? Perhaps disingenuous to call "our web app" This is the heavy lifting: https://github.com/apple/ml-sharp Previous discussion: https://news.ycombinator.com/item?id=46284658

Yeah I think you're right. It calls that out (in really tiny footer text) that it's leveraging ml-sharp.

It's pretty trivial to get running locally and generating the PLY files. Spark's a pretty good renderer for it after you've generated the gaussian splats.

https://github.com/sparkjsdev/spark

Re: Turn a single image into a navigable 3D Gaussian Splat with depth

#28

Post by esafak »

voodooEntity wrote: post #21
smusamashah wrote: post #18

If this model is so good at estimating depth from single image, shouldn't it also be able to take multiple images as input and estimate even better? But searching a bit it looks like this is supposed to be a single image to 3D only. I don't understand why it does not (can not?) work with multiple images.

If you have multiple images you could use photogrammetry. At the end, if you want to "fill in the blanks" llm will always "make up" stuff, based on all of its training data. With a technology like photogrammetry you can get much better results, therefor if you have multiple angled images and dont really need to make up stuff, its better to use such

Surely this is not an LLM?

Re: Turn a single image into a navigable 3D Gaussian Splat with depth

#29

Post by verytrivial »

mawadev wrote: post #10

Stuck at 90% forever..

My understanding of JavaScript is cursory, but my reading of that webpage is the UI is just smoke and mirrors, and it is just waiting for the whole thing to be processed in a single remote API call to some back-end system. If the back-end is down, it will always stop at 90%. The crawling progress bar is fake with canned messages updated with Math.Random() delays. Gives you something to look at, I guess, but seems a little misleading. Might be wrong ...
Post reply on HN