Live data from Hacker News

Gaussian Splatting – A$AP Rocky "Helicopter" music video

radiancefields.com

71–80 of 278 posts

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#71

How did Rhianna look him in the eyes and say "yes babe, good album, release it, this is what the people wanted after 7 years, it is pleasing to listen to and enjoyable"?

Im sure it was more like, “hey babe, can I get a few millions to go in the studio and experiment/make some art?” And then she was like, “yeah go for it! Make some weird shit.”

If I was in his position I’d probably be doing the same. Why bother with another top hit that pleases the masses.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#72

Earlier quoted context omitted.

From the article: >Evercoast deployed a 56 camera RGB-D array Do you know which depth cameras they used?

Couldn’t you just use iphone pros for this? I developed an app specifically for photogrammetry capture using AR and the depth sensor as it seemed like a cheap alternative. EDIT: I realize a phone is not on the same level as a red camera, but i just saw iphones as a massively cheaper option to alternatives in the field i worked in.

I think it's because they already had proven capture hardware, harvest, and processing workflows.

But yes, you can easily use iPhones for this now.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#75
post #6

To be honest it looks like it was rendered in an old version of Unreal Engine. That may be an intentional choice - I wonder how realistic guassian splatting can look? Can you redo lights, shadows, remove or move parts of the scene, while preserving the original fidelity and realism? The way TV/movie production is going (record 100s of hours of footage from multiple angles and edit it all in post) I wonder if this is…

For me it felt more like higher detail version of Teardown, the voxel-based 3d demolition game. Sure it's splats and not voxels, but the camera and the lighting give this strong voxel game vibe.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#77
post #16
post #6

To be honest it looks like it was rendered in an old version of Unreal Engine. That may be an intentional choice - I wonder how realistic guassian splatting can look? Can you redo lights, shadows, remove or move parts of the scene, while preserving the original fidelity and realism? The way TV/movie production is going (record 100s of hours of footage from multiple angles and edit it all in post) I wonder if this is…

Yes, they talk about this in the article and that’s exactly what they did.

It wasn't clear to me how much this was intentional vs. being the limits of the technology at the moment.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#78

Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?

1. Create a point cloud from a scene (either via lidar, or via photogrammetry from multiple images)

2. Replace each point of the point cloud with a fuzzy ellipsoid, that has a bunch of parameters for its position + size + orientation + view-dependent color (via spherical harmonics up to some low order)

3. If you render these ellipsoids using a differentiable renderer, then you can subtract the resulting image from the ground truth (i.e. your original photos), and calculate the partial derivatives of the error with respect to each of the millions of ellipsoid parameters that you fed into the renderer.

4. Now you can run gradient descent using the differentiable renderer, which makes your fuzzy ellipsoids converge to something closely reproducing the ground truth images (from multiple angles).

5. Since the ellipsoids started at the 3D point cloud's positions, the 3D structure of the scene will likely be preserved during gradient descent, thus the resulting scene will support novel camera angles with plausible-looking results.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#79

Earlier quoted context omitted.

Couldn’t you just use iphone pros for this? I developed an app specifically for photogrammetry capture using AR and the depth sensor as it seemed like a cheap alternative. EDIT: I realize a phone is not on the same level as a red camera, but i just saw iphones as a massively cheaper option to alternatives in the field i worked in.

Why would they go for the cheapest option?

It was more the point that technology is much cheaper. The company i worked for had completely missed it while trying to develop in house solutions.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#80
post #20

Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?

Gaussian splatting is a way to record 3-dimensional video. You capture a scene from many angles simultaneously and then combine all of those into a single representation. Ideally, that representation is good enough that you can then, post-production, simulate camera angles you didn't originally record. For example, the camera orbits around the performers in this music video are difficult to imagine in real space. Eve…

> Gaussian splatting is a way to record 3-dimensional video.

I would say it's a 3D photo, not a 3D video. But there are already extensions to dynamic scenes with movement.

Post reply on HN