LiveSplat is a system for turning RGBD camera streams into Gaussian splat scenes in real-time. The system works by passing all the RGBD frames into a feed forward neural net that outputs the current scene as Gaussian splats. These splats are then rendered in real-time. I've put together a demo video at the link above.
Show HN: Real-Time Gaussian Splatting
1–10 of 61 posts
Re: Show HN: Real-Time Gaussian Splatting
#2This is realtime capture/display? Presumable (at this stage) for local viewing? Is that right?
Re: Show HN: Real-Time Gaussian Splatting
#3This is realtime capture/display? Presumable (at this stage) for local viewing? Is that right?
Yes realtime capture and display. Locality is not required. You can send the source RGBD video streams over IP and in fact I have that component working in the larger codebase that this was split off from. For that use case, you need to do some sort of compression. The RGB stream compression is a pretty solved problem, but the depth channel needs special consideration since "perceptual loss" in the depth space is not a well researched area.
Re: Show HN: Real-Time Gaussian Splatting
#4nice
Re: Show HN: Real-Time Gaussian Splatting
#5OP, this is incredible. I worry that people might see a "glitchy 3D video" and might not understand the significance of this.
This is getting unreal. They're becoming fast and high fidelity. Once we get better editing capabilities and can shape the Gaussian fields, this will become the prevailing means of creating and distributing media.
Turning any source into something 4D volumetric that you can easily mold as clay, relight, reshape. A fully interactable and playable 4D canvas.
Imagine if the work being done with diffusion models could read and write from Gaussian fields instead of just pixels. It could look like anything: real life, Ghibli, Pixar, whatever.
I can't imagine where this tech will be in five years.
Re: Show HN: Real-Time Gaussian Splatting
#6OP, this is incredible. I worry that people might see a "glitchy 3D video" and might not understand the significance of this. This is getting unreal. They're becoming fast and high fidelity. Once we get better editing capabilities and can shape the Gaussian fields, this will become the prevailing means of creating and distributing media. Turning any source into something 4D volumetric that you can easily mold as clay…
Thanks so much! Even when I was putting together the demo video I was getting a little self-critical about the visual glitches. But I agree the tech will get better over time. I imagine we will be able to have virtual front row seats at any live event, and many other applications we haven't thought of yet.
Re: Show HN: Real-Time Gaussian Splatting
#7OP, this is incredible. I worry that people might see a "glitchy 3D video" and might not understand the significance of this. This is getting unreal. They're becoming fast and high fidelity. Once we get better editing capabilities and can shape the Gaussian fields, this will become the prevailing means of creating and distributing media. Turning any source into something 4D volumetric that you can easily mold as clay…
I know enough about 3D rendering to know that Gaussian splatting's one of the Big New Things in high-performance rendering, so I understand that this is a big deal -- but I can't quantify why, or how big a deal it is.
Could you or someone else wise in the ways of graphics give me a layperson's rundown of how this works, why it's considered so important, and what the technical challenges are given that an RGB+D(epth?) stream is the input?
Re: Show HN: Real-Time Gaussian Splatting
#8Please excuse my naive question - isn't Gaussian Splatting usually used to create 3D imagery from 2D? How does providing 3D input data make sense in this context?
Re: Show HN: Real-Time Gaussian Splatting
#9OP, this is incredible. I worry that people might see a "glitchy 3D video" and might not understand the significance of this. This is getting unreal. They're becoming fast and high fidelity. Once we get better editing capabilities and can shape the Gaussian fields, this will become the prevailing means of creating and distributing media. Turning any source into something 4D volumetric that you can easily mold as clay…
Thanks so much! Even when I was putting together the demo video I was getting a little self-critical about the visual glitches. But I agree the tech will get better over time. I imagine we will be able to have virtual front row seats at any live event, and many other applications we haven't thought of yet.
> I imagine we will be able to have virtual front row seats at any live event, and many other applications we haven't thought of yet.
100%. And style-transfer it into steam punk or H.R. Giger or cartoons or anime. Or dream up new fantasy worlds instantaneously. Explore them, play them, shape them like Minecraft-becomes-holodeck. With physics and tactile responses.
I'm so excited for everything happening in graphics right now.
Keep it up! You're at the forefront!
Re: Show HN: Real-Time Gaussian Splatting
#10[flagged]