Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gaussian Splatting – A$AP Rocky "Helicopter" music video

radiancefields.com

Post reply on HN
Print view View original

141–150 of 278 posts

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#141

Post by darhodester »

henjodottech wrote: post #120
darhodester wrote: post #39

Hi, I'm David Rhodes, Co-founder of CG Nomads, developer of GSOPs (Gaussian Splatting Operators) for SideFX Houdini. GSOPs was used in combination with OTOY OctaneRender to produce this music video. If you're interested in the technology and its capabilities, learn more at https://www.cgnomads.com/ or AMA. Try GSOPs yourself: https://github.com/cgnomads/GSOPs (example content included).

I’m fascinated by the aesthetic of this technique. I remember early versions that were completely glitched out and presented 3d clouds of noise and fragments to traverse through. I’m curious if you have any thoughts about creatively ‘abusing’ this tech? Perhaps misaligning things somehow or using some wrong inputs.

There's a ton of fun tricks you can perform with Gaussian splatting!

You're right that you can intentionally under-construct your scenes. These can create a dream-like effect.

It's also possible to stylize your Gaussian splats to produce NPR effects. Check out David Lisser's amazing work: https://davidlisser.co.uk/Surface-Tension.

Additionally, you can intentionally introduce view-dependent ghosting artifacts. In other words, if you take images from a certain angle that contain an object, and remove that object for other views, it can produce a lenticular/holographic effect.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#142

Post by suzzer99 »

shwaj wrote: post #50
suzzer99 wrote: post #38

Earlier quoted context omitted.

Can you put "Gaussing splats" in some kind of real world metaphor so I can understand what it means? Either that or explain why "Gaussian" and why "splat". I am vaguely aware of stuff like Gaussian blur on Photoshop. But I never really knew what it does.

How can you expect someone to tailor a custom explanation, when they don’t know your level of mathematical understanding, or even your level of curiosity. You don’t know what a Gaussian blur does; do you know what a Gaussian is? How deeply do you want to understand? If you’re curious start with the Wikipedia article and use an LLM to help you understand the parts that don’t make sense. Or just ask the LLM to provide…

> How can you expect someone to tailor a custom explanation, when they don’t know your level of mathematical understanding, or even your level of curiosity.

The other two replies did a pretty good job!

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#143

Post by darhodester »

henjodottech wrote: post #120
darhodester wrote: post #39

Hi, I'm David Rhodes, Co-founder of CG Nomads, developer of GSOPs (Gaussian Splatting Operators) for SideFX Houdini. GSOPs was used in combination with OTOY OctaneRender to produce this music video. If you're interested in the technology and its capabilities, learn more at https://www.cgnomads.com/ or AMA. Try GSOPs yourself: https://github.com/cgnomads/GSOPs (example content included).

I’m fascinated by the aesthetic of this technique. I remember early versions that were completely glitched out and presented 3d clouds of noise and fragments to traverse through. I’m curious if you have any thoughts about creatively ‘abusing’ this tech? Perhaps misaligning things somehow or using some wrong inputs.

The ghost effect is pretty cool, too! https://www.youtube.com/watch?v=DQGtimwfpIo

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#144

Post by darhodester »

secretsatan wrote: post #104
darhodester wrote: post #72

Earlier quoted context omitted.

I think it's because they already had proven capture hardware, harvest, and processing workflows. But yes, you can easily use iPhones for this now.

Looks great by the way, i was wondering if there’s a file format for volumetric video captures

Some companies have a proprietary file format for compressed 4D Gaussian splatting. For example: https://www.gracia.ai and https://www.4dv.ai.

Check this project, for example: https://zju3dv.github.io/freetimegs/

Unfortunately, these formats are currently closed behind cloud processing so adoption is a rather low.

Before Gaussian splatting, textured mesh caches would be used for volumetric video (e.g. Alembic geometry).

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#145

Post by michaelrubloff »

IshKebab wrote: post #127
darhodester wrote: post #32

Earlier quoted context omitted.

Gaussian splatting is not NeRF (neural radiance field), but it is a type of radiance field, and supports novel view synthesis. The difference is in an explicit point cloud representation (Gaussian splatting), versus a process that needs to be inferred by a neural network.

It's not a type of radiance field.

It’s literally the name of gaussian splatting. 3D Gaussian Splatting for Real Time Radiance Fields

https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#148

Post by bredren »

bininunez wrote: post #106
sbierwagen wrote: post #43

Earlier quoted context omitted.

From the article: >Evercoast deployed a 56 camera RGB-D array Do you know which depth cameras they used?

We (Evercoast) used 56 RealSense D455s. Our software can run with any camera input, from depth cameras to machine vision to cinema REDs. But for this, RealSense did the job. The higher end the camera, the more expensive and time consuming everything is. We have a cloud platform to scale rendering, but it’s still overall more costly (time and money) to use high res. We’ve worked hard to make even low res data look awe…

You may have explained this elsewhere, but if not—-what kind of post processing did you do to upscale or refine the realsense video?

Can you add any interesting details on the benchmarking done against the RED camera rig?

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#149

Post by zapzupnz »

throwadobe wrote: post #146
nodra wrote: post #140

Earlier quoted context omitted.

What do you mean by that?

[flagged]

Not everything has to be. Sometimes, an artist's style or a particular track just hits a particular vibe one may be after or need in a particular moment.

I'm not a fan of this music either but I could imagine hearing it while I'm studying or coding.

Don't trash something just because it's not your vibe. Not everything has to be Mozart.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#150

Post by ravedave5 »

pleurotus wrote: post #8

Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?

This is a REALLY good video explaining it. https://www.youtube.com/watch?v=eekCQQYwlgA
Post reply on HN