Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gaussian Splatting – A$AP Rocky "Helicopter" music video

radiancefields.com

Post reply on HN
Print view View original

111–120 of 278 posts

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#111

Post by michaelrubloff »

rkuykendall-com wrote: post #13
pleurotus wrote: post #8

Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?

I found this VFX breakdown of the recent Superman movie to have a great explanation of what it is and what it makes possible: https://youtu.be/eyAVWH61R8E?t=232 tl;dr eli5: Instead of capturing spots of color as they would appear to a camera, they capture spots of color and where they exist in the world. By combining multiple cameras doing this, you can make a 3D works from footage that you can then zoom a virtual ca…

I also spoke to the vfx team from Superman on how they achieved the reconstructions! (I’m also the author for the Helicopter article here).

https://radiancefields.com/gaussian-splatting-in-superman

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#112

Post by numpad0 »

secretsatan wrote: post #55
sbierwagen wrote: post #43

Earlier quoted context omitted.

From the article: >Evercoast deployed a 56 camera RGB-D array Do you know which depth cameras they used?

Couldn’t you just use iphone pros for this? I developed an app specifically for photogrammetry capture using AR and the depth sensor as it seemed like a cheap alternative. EDIT: I realize a phone is not on the same level as a red camera, but i just saw iphones as a massively cheaper option to alternatives in the field i worked in.

A single camera only captures the side of the object facing the camera. Knowing how far away that camera facing side of a Rubik's Cube help if you were making educated guesses(novel view synthesis), but it won't solve the problem of actually photographing the backside.

There are usually six sides on a cube, which means you need minimum six iPhone around an object to capture all sides of it to be able to then freely move around it. You might as well seek open-source alternatives than relying on Apple surprise boxes for that.

In cases where your subject would be static, such as it being a building, then you can wave around a single iPhone for the same effect for a result comparable to more expensive rigs, of course.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#113

Post by michaelrubloff »

noman-land wrote: post #7

Really amazing video. Unfortunately this article is like 60% over my head. Regardless, I actually love reading jargon-filled statements like this that are totally normal to the initiated but are completely inscrutable to outsiders. "That data was then brought into Houdini, where the post production team used CG Nomads GSOPs for manipulation and sequencing, and OTOY’s OctaneRender for final rendering. Thanks to this c…

My bad! I am the author. Gaussian splatting allows you to take a series of normal 2D images or a video and reconstruct very lifelike 3D from it. It’s a type of radiance field, like NeRFs or voxel based methods like Plenoxels!

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#114

Post by michaelrubloff »

rjh29 wrote: post #6

To be honest it looks like it was rendered in an old version of Unreal Engine. That may be an intentional choice - I wonder how realistic guassian splatting can look? Can you redo lights, shadows, remove or move parts of the scene, while preserving the original fidelity and realism? The way TV/movie production is going (record 100s of hours of footage from multiple angles and edit it all in post) I wonder if this is…

We will be able to have imax level 3D technically today if you feed it the correct data

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#115

Post by darhodester »

jeffgreco wrote: post #110
darhodester wrote: post #39

Hi, I'm David Rhodes, Co-founder of CG Nomads, developer of GSOPs (Gaussian Splatting Operators) for SideFX Houdini. GSOPs was used in combination with OTOY OctaneRender to produce this music video. If you're interested in the technology and its capabilities, learn more at https://www.cgnomads.com/ or AMA. Try GSOPs yourself: https://github.com/cgnomads/GSOPs (example content included).

Great work! I’d love to see a proper BTS or case study.

I do believe a BTS is being developed.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#116

Post by krackers »

pleurotus wrote: post #8

Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?

https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr... and for a visual demo of the result https://antimatter15.com/splat/

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#118

Post by periodjet »

The end result is really interesting. As others have pointed out, it looks sort of like it was rendered by an early 2000s game engine. There’s a cohesiveness to the art direction that you just can’t get from green screens and the like. In service of some of the worst music made by human brains, but still really cool tech.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#119

Post by mlyle »

hamburglar wrote: post #30
londons_explore wrote: post #9

Pretty sure most of this could be filmed with a camera drone and preprogrammed flight path... Did the Gaussian splatting actually make it any cheaper? Especially considering that it needed 50+ fixed camera angles to splat properly, and extensive post-processing work both computationally and human labour, a camera drone just seems easier.

> Pretty sure most of this could be filmed with a camera drone and preprogrammed flight path This is a “Dropbox is just ftp and rsync” level comment. There’s a shot in there where Rocky is sitting on top of the spinning blades of a helicopter and the camera smoothly transitions from flying around the room to solidly rotating along with the blades, so it’s fixed relative to rocky. Not only would programming a camera d…

Drones and 2d compositing could do a lot. They would excel in some areas used in the video, require far more resources than this technique in others, and be completely infeasible on a few.

They would look much better in a very "familiar" way. They would have much less of the glitch and dynamic aesthetic that makes this so novel.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#120

Post by henjodottech »

darhodester wrote: post #39

Hi, I'm David Rhodes, Co-founder of CG Nomads, developer of GSOPs (Gaussian Splatting Operators) for SideFX Houdini. GSOPs was used in combination with OTOY OctaneRender to produce this music video. If you're interested in the technology and its capabilities, learn more at https://www.cgnomads.com/ or AMA. Try GSOPs yourself: https://github.com/cgnomads/GSOPs (example content included).

I’m fascinated by the aesthetic of this technique. I remember early versions that were completely glitched out and presented 3d clouds of noise and fragments to traverse through. I’m curious if you have any thoughts about creatively ‘abusing’ this tech? Perhaps misaligning things somehow or using some wrong inputs.
Post reply on HN