Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SMERF: Streamable Memory Efficient Radiance Fields

smerf-3d.github.io

Post reply on HN
Print view View original

31–40 of 152 posts

Re: SMERF: Streamable Memory Efficient Radiance Fields

#31

Post by chpatrick »

nightpool wrote: post #14
barrkel wrote: post #13

The mirror on the wall of the bathroom in the Berlin location looks through to the kitchen in the next room. I guess the depth gauging algorithm uses parallax, and mirrors confuse it, seeming like windows. The kitchen has a blob of blurriness as the rear of the mirror intrudes into kitchen, but you can see through the blurriness to either room. The effect is a bit spooky. I felt like a ghost going through walls.

The refigerator in the NYC scene has a very slick specular lighting effect based on the angle you're viewing it from, and if you go "into" the fridge you can see it's actually generating a whole 3d scene with blurry grey and white colors that turn out to precisely mimic the effects of the light from the windows bouncing off the metal, and you can look "out" from the fridge into the rest of the room. Same as the full-…

This happens with any 3D reconstruction. It's because any mirror is indistinguishable from a window into a mirrored room. The tricky thing is if there's actually a something behind the mirror as well.

Re: SMERF: Streamable Memory Efficient Radiance Fields

#32

Post by vyrotek »

VikingCoder wrote: post #16

Wow. Some questions: Take for instance the fulllivingroom demo. (I prefer fps mode.) 1) How many images are input? 2) How long does it take to compute these models? 3) How long does it take to prepare these models for this browser, with all levels, etc? 4) Have you tried this in VR yet?

Not exactly what you asked for. But I recently came across this VR example using Gaussian Splatting instead. Exciting times.

https://twitter.com/gracia_vr/status/1731731549886787634

https://www.gracia.ai

Re: SMERF: Streamable Memory Efficient Radiance Fields

#39

Post by doctoboggan »

catskul2 wrote: post #30

When might we see this in consumer VR? I'm surprised we don't already but I was suspecting it was a computation constraint. Does this relieve the computation constraint enough to run on Quest 2/3? Is there something else that would prevent binocular use?

I recently got a new quest and I am wondering the same thing. The fact that this is currently running in a browser (and can run on a mobile device) gives me hope that we will see something like this in VR sooner rather than later.

Re: SMERF: Streamable Memory Efficient Radiance Fields

#40

Post by duckworthd »

promiseofbeans wrote: post #15

It runs impressively well on my 2yo s21fe. It was super impressive how it streamed in more images as I explored the space. The tv reflections in the Berlin demo were super impressive. My one note is that it look a really long time to load all the images - the scene wouldn't render until all ~40 initial images loaded. Would it be possible to start partially rendering as the images arrive, or do you need to wait for al…

Pardon our dust: "images" is a bad name for what's being loaded. Past versions of this approach (MERF) stored feature vectors in PNG images. We replace them with binary arrays. Unfortunately, all such arrays need to be loaded before the first frame can be rendered.

You do however point out one weakness of SMERF: large payload sizes. If we can figure out how to compress them by 10x, it'll be a very different experience!

Post reply on HN