Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gaussian Splatting – A$AP Rocky "Helicopter" music video

radiancefields.com

Post reply on HN
Print view View original

251–260 of 278 posts

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#251

Post by bininunez »

darhodester wrote: post #245
echelon wrote: post #195

Earlier quoted context omitted.

Y'all did such a good job with this. It captivated HN and was the top post for the entire day, and will probably last for much of tomorrow. If you don't know already, you need to leverage this. HN is one of the biggest channels of engineers and venture capitalists on the internet. It's almost pure signal (minus some grumpy engineer grumblings - we're a grouchy lot sometimes). Post your contract info here. You might g…

First, all credit for execution and vision of Helicopter go to A$AP, Dan Streit, and Grin Machine ( https://www.linkedin.com/company/grin-machine/about/ ). Evercoast and Wild Capture were also involved. Second, it's very motivating to read this! My background is in video game development (only recently transitioning to VFX). My dream is to make a Gaussian splatting content creation and game development platform with…

Thanks darhodester! It was definitely a broad team effort that started with Rocky and Streit's creative genius which was then made possible by Evercoast's software to capture and generate all the 4D splat data (www.evercoast.com), which then flowed to the incredible people at Grin Machine and Wild capture who used GSOPs and OctaneRender.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#253

Post by Syzygies »

tamat wrote: post #200
darhodester wrote: post #39

Hi, I'm David Rhodes, Co-founder of CG Nomads, developer of GSOPs (Gaussian Splatting Operators) for SideFX Houdini. GSOPs was used in combination with OTOY OctaneRender to produce this music video. If you're interested in the technology and its capabilities, learn more at https://www.cgnomads.com/ or AMA. Try GSOPs yourself: https://github.com/cgnomads/GSOPs (example content included).

nice work. I can see that relighting is still a work in progress, as the virtual spot lights tends to look flat and fake. I understand that you are just making brighter splats that fall inside the spotlight cone and darker the ones behind lots of splats. Do you know if there are plans for gaussian splats to capture unlit albedo, roughness and metalness? So we can relight in a more realistic manner? Also, environment…

Back in 2001 I was the math consultant for "A Beautiful Mind". One spends a lot of time waiting on a film set. Eventually one wonders why.

The majority of wait time was the cinematographer lighting each scene. I imagined a workflow where secondary digital cameras captured 3D information, and all lighting took place in post production. Film productions hemorrhage money by the second; this would be a massive cost saving.

I described this idea to a venture capitalist friend, who concluded one already needed to be a player to pull this off. I mentioned this to an acquaintance at Pixar (a logical player) and they went silent.

Still, we don't shoot movies this way. Not there yet...

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#254

Post by hbarka »

> Viewers assume the imagery is AI-generated.

Watching this Helicopter music video made me recall a scene in Money for Nothing by Dire Straits, which was famously the first music video to air on MTV Europe (when the MTV phenomenon launched the 80s). It used 3D animation for the human characters and was considered groundbreaking animation in that time. The irony is we knew it was computer generated but now human generated is indistinguishable from AI.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#255

Post by darhodester »

tamat wrote: post #200
darhodester wrote: post #39

Hi, I'm David Rhodes, Co-founder of CG Nomads, developer of GSOPs (Gaussian Splatting Operators) for SideFX Houdini. GSOPs was used in combination with OTOY OctaneRender to produce this music video. If you're interested in the technology and its capabilities, learn more at https://www.cgnomads.com/ or AMA. Try GSOPs yourself: https://github.com/cgnomads/GSOPs (example content included).

nice work. I can see that relighting is still a work in progress, as the virtual spot lights tends to look flat and fake. I understand that you are just making brighter splats that fall inside the spotlight cone and darker the ones behind lots of splats. Do you know if there are plans for gaussian splats to capture unlit albedo, roughness and metalness? So we can relight in a more realistic manner? Also, environment…

This may interest you: https://www.linkedin.com/posts/radiancefields_in-case-you-we...

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#256

Post by corysama »

cpt_sobel wrote: post #232
meindnoch wrote: post #78

Earlier quoted context omitted.

1. Create a point cloud from a scene (either via lidar, or via photogrammetry from multiple images) 2. Replace each point of the point cloud with a fuzzy ellipsoid, that has a bunch of parameters for its position + size + orientation + view-dependent color (via spherical harmonics up to some low order) 3. If you render these ellipsoids using a differentiable renderer, then you can subtract the resulting image from th…

And what about the "mature enough" part? How has it changed / progressed recently?

The field is advancing rapidly. New research papers are being published daily for a few years now. The best news feed I've found on the topic is

https://radiancefields.com/

https://x.com/RadianceFields alt: https://xcancel.com/RadianceFields

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#257

Post by bininunez »

Hello! I'm Ben Nunez, CEO at Evercoast. Our software was used to capture and reconstruct the 4D Gaussian splats in this A$AP Rocky video.

The music video is a mix of creative and technical genius from several different teams, and it's ultimately a byproduct of building tooling to capture reality once and reuse it downstream.

There’s a lot more to explore here. Once you have grounded 4D human motion and appearance tied to a stable world coordinate system, it becomes a missing primitive for things like world models, simulation, and embodied AI, where synthetic or purely parametric humans tend to break down.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#258

Post by nkotov »

prmoustache wrote: post #206
filoleg wrote: post #52

Earlier quoted context omitted.

Similarly, the music video for Taylor Swif[0] (another track by A$AP Rocky) is just as surrealistic and weird in the best way possible, but with an eastern european flavor of it (which is obviously intentional and makes sense, given the filming location and being very on-the-nose with the theme). 0. https://youtu.be/5URefVYaJrA

I can see how this kind of videos can attract the tiktok addicts with less than 3 seconds of attention time. I wonder what will be the state of cinema/series/video clips in 30 years? Will singers/rappers give up sentences completely and just mention names of emojis? Will we have to use screens at 576hz to be able to watch acclerated videos without seeing a constant blur? I guess most kids from today would fall asleep…

I still rewatch this video once or twice a year. I think it's a classic in my opinion.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#259

Post by GrowingSideways »

darhodester wrote: post #250
GrowingSideways wrote: post #209

Earlier quoted context omitted.

Take the money and never admit to selling this shit. Why would you ever willingly associate your name with this?

Read the room. Plenty of people are interested in the aesthetics and the technology.

Just because people want to give you money doesn't mean you toss your dignity out the window.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#260

Post by corysama »

MITSardine wrote: post #234
tel wrote: post #20

Earlier quoted context omitted.

Gaussian splatting is a way to record 3-dimensional video. You capture a scene from many angles simultaneously and then combine all of those into a single representation. Ideally, that representation is good enough that you can then, post-production, simulate camera angles you didn't originally record. For example, the camera orbits around the performers in this music video are difficult to imagine in real space. Eve…

Are meshes not used instead of gaussian splats only due to robustness reasons? I.e., if there were a piece of software that could reliably turn a colored point cloud into a textured mesh, would that be preferable?

Photogrammetry has been around for a long time now. It uses pretty much the same inputs to create meshes from a collection of images of a scene.

It works well for what it does. But, it's mostly only effective for opaque, diffuse, solid surfaces. It can't handle transparency, reflection or "fuzz". Capturing material response is possible, but requires expensive setups.

A scene like this poodle https://superspl.at/view?id=6d4b84d3 or this bee https://superspl.at/view?id=cf6ac78e would be pretty much impossible with photogrammetry and very difficult with manual, traditional, polygon workflows. Those are not videos. Spin them around.

Post reply on HN