Live data from Hacker News

SMERF: Streamable Memory Efficient Radiance Fields

smerf-3d.github.io

11–20 of 152 posts

Re: SMERF: Streamable Memory Efficient Radiance Fields

#11
This is __really__ stunning work, huge, huge, deal that I'm seeing this in a web browser on my phone. Congratulations!

When I look at the NYC scene in the highest quality on desktop, I'm surprised by how low-quality ex. the stuff on the counter and shelves is. So then I load the lego model, and see that's _very_ detailed, so it doesn't seem inherent to the method.

Is it a consequence of input photo quality, or something else?

Re: SMERF: Streamable Memory Efficient Radiance Fields

#13
The mirror on the wall of the bathroom in the Berlin location looks through to the kitchen in the next room. I guess the depth gauging algorithm uses parallax, and mirrors confuse it, seeming like windows. The kitchen has a blob of blurriness as the rear of the mirror intrudes into kitchen, but you can see through the blurriness to either room.

The effect is a bit spooky. I felt like a ghost going through walls.

Re: SMERF: Streamable Memory Efficient Radiance Fields

#14
post #13

The mirror on the wall of the bathroom in the Berlin location looks through to the kitchen in the next room. I guess the depth gauging algorithm uses parallax, and mirrors confuse it, seeming like windows. The kitchen has a blob of blurriness as the rear of the mirror intrudes into kitchen, but you can see through the blurriness to either room. The effect is a bit spooky. I felt like a ghost going through walls.

The refigerator in the NYC scene has a very slick specular lighting effect based on the angle you're viewing it from, and if you go "into" the fridge you can see it's actually generating a whole 3d scene with blurry grey and white colors that turn out to precisely mimic the effects of the light from the windows bouncing off the metal, and you can look "out" from the fridge into the rest of the room. Same as the full-length mirror in the bedroom in the same scene—there's a whole virtual "mirror room" that's been built out behind the mirror to give the illusion of depth as you look through it. Very cool and unique consequence of the technology

Re: SMERF: Streamable Memory Efficient Radiance Fields

#15
It runs impressively well on my 2yo s21fe. It was super impressive how it streamed in more images as I explored the space. The tv reflections in the Berlin demo were super impressive.

My one note is that it look a really long time to load all the images - the scene wouldn't render until all ~40 initial images loaded. Would it be possible to start partially rendering as the images arrive, or do you need to wait for all of them before you can do the first big render?

Re: SMERF: Streamable Memory Efficient Radiance Fields

#16
Wow. Some questions:

Take for instance the fulllivingroom demo. (I prefer fps mode.)

1) How many images are input?

2) How long does it take to compute these models?

3) How long does it take to prepare these models for this browser, with all levels, etc?

4) Have you tried this in VR yet?

Re: SMERF: Streamable Memory Efficient Radiance Fields

#18
post #13

The mirror on the wall of the bathroom in the Berlin location looks through to the kitchen in the next room. I guess the depth gauging algorithm uses parallax, and mirrors confuse it, seeming like windows. The kitchen has a blob of blurriness as the rear of the mirror intrudes into kitchen, but you can see through the blurriness to either room. The effect is a bit spooky. I felt like a ghost going through walls.

The refigerator in the NYC scene has a very slick specular lighting effect based on the angle you're viewing it from, and if you go "into" the fridge you can see it's actually generating a whole 3d scene with blurry grey and white colors that turn out to precisely mimic the effects of the light from the windows bouncing off the metal, and you can look "out" from the fridge into the rest of the room. Same as the full-…

Wow, thanks for the tip. Fridge reflection world is so cool. Feels like something David Lynch might dream up.

A girl is eating her morning cereal. Suddenly she looks apprehensively at the fridge. Camera dollies towards the appliance and seamlessly penetrates the reflective surface, revealing a deep hidden space that exactly matches the reflection. At the dark end of the tunnel, something stirs... A wildly grinning man takes a step forward and screams.

Re: SMERF: Streamable Memory Efficient Radiance Fields

#19
post #13

The mirror on the wall of the bathroom in the Berlin location looks through to the kitchen in the next room. I guess the depth gauging algorithm uses parallax, and mirrors confuse it, seeming like windows. The kitchen has a blob of blurriness as the rear of the mirror intrudes into kitchen, but you can see through the blurriness to either room. The effect is a bit spooky. I felt like a ghost going through walls.

The refigerator in the NYC scene has a very slick specular lighting effect based on the angle you're viewing it from, and if you go "into" the fridge you can see it's actually generating a whole 3d scene with blurry grey and white colors that turn out to precisely mimic the effects of the light from the windows bouncing off the metal, and you can look "out" from the fridge into the rest of the room. Same as the full-…

Neat! Here are some screenshots of the same phenomenon with the TV in Berlin: https://imgur.com/a/3zAA5K8
Post reply on HN