Wow this is insanely cool.
If you make it work within ThreeJS, you're going to leave a trace in the history of 3D on the web with that stuff!
Show HN: Real-Time 3D Gaussian Splatting in WebGL
41–50 of 64 posts
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#42This is really cool! The control scheme is confusing though. Instead of the typical WASD for moving and using the mouse to look around, dragging the mouse moves forwards and backwards and orbits around some point, A and D strafe, while W and S look up and down. EDIT: Looks like a full list of controls is in the readme: https://github.com/antimatter15/splat#controls
Author here- I'm sorry about the camera controls! Happy to accept pull requests that replace it with something more sensible The original idea was to be able to navigate around with just arrow keys (conceptually by turning yourself around in place and being able to walk back and forward).
This is insanely cool!
If you integrate this with ThreeJS you'd have a lot of control options for free!
Whilst you're here, I have a question for you: It seems like you don't render read gaussians (I see sharp edges in many cases). Is this a bug on my side or is this an optimization made to be able to run fast? I created an issue to discuss if you prefer https://github.com/antimatter15/splat/issues/2
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#43Can't wait to pull this up on my desktop tomorrow.
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#44Gotta try on a pc for some reason on iOS the cloudiness feels more like nerf than Gaussian to me for some reason, gotta try it on pc later
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#45This is beyond cool. Point clouds are one thing but this… this is amazing. Kudos and great job. It even runs on my work Lenovo at 60fps.
It runs on my mid range phone at 36fps. Did not expect that.
Lots of artefacts though, especially if I move the camera.
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#46This is beyond cool. Point clouds are one thing but this… this is amazing. Kudos and great job. It even runs on my work Lenovo at 60fps.
It runs on my mid range phone at 36fps. Did not expect that. Lots of artefacts though, especially if I move the camera.
those artifacts are part of the algorithm.
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#47This is really cool! The control scheme is confusing though. Instead of the typical WASD for moving and using the mouse to look around, dragging the mouse moves forwards and backwards and orbits around some point, A and D strafe, while W and S look up and down. EDIT: Looks like a full list of controls is in the readme: https://github.com/antimatter15/splat#controls
FWIW, OP, I liked the control scheme a lot (using mouse only).
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#48Very impressive! Curious what the frame rate would be like for stereoscopic rendering of the same scene on the same hardware. Are there optimizations to be had past the halfway mark?
Definitely. One of the time-consuming parts of rendering is sorting the gaussians by distance to the camera, which for two nearby cameras could be optimized. This also goes for adjacent frames - assuming smooth motion im pretty sure there is some speedup to be had by assuming the previous sort will be close to correct rather than starting from scratch each frame.
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#49Earlier quoted context omitted.
A thought occurred to me: is this similar to how Media Molecule's Dreams on the PS4/5 renders its scenes?
> Media Molecule “Dreams” has a splat-based renderer (I think the shipped version is not purely splat-based but a combination of several techniques). From: https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr... Good eye
Also maybe worth mentioning that the 2nd author of the InstantNGP paper is also a cofounder of and lead tech guy at Media Molecule, Alex Evans. I've been a huge fan of his work since 90s demoscene and briefly got to work with him at Lionhead Studios, guy's a legit genius.
Re: Show HN: Real-Time 3D Gaussian Splatting in WebGL
#50Earlier quoted context omitted.
Gaussian splatting is a fancy word for pointcloud but with coloured shapes instead of points. Its been around for ages, but It was never used because if you have a million points in a point cloud, you'd need to artistically manipulate a million points. Its like 3d hair, its pretty simple, just render a billion hairs, but in practice its hard to make it look good. Here we tell a machine learning model to adjust the an…
It's a little bit more than that. Gaussians are view-dependent, which means that they can capture the full radiance field of the scene, rather than just the color and geometry of the objects. All the light bouncing around from different objects can be reproduced, including reflections etc. See the reflections here: https://www.youtube.com/watch?v=mD0oBE9LJTQ This is also pretty good, but more subtle: https://www.yout…
This implementation does not support view-dependence though (mentioned in the readme)