Live data from Hacker News

Gaussian Splatting – A$AP Rocky "Helicopter" music video

radiancefields.com

231–240 of 278 posts

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#231
post #225

Earlier quoted context omitted.

I guess the technology still has some quality limitations, otherwise we would already see it in mainstream movies, e.g. to simulate smooth camera motions beyond what is achievable with video stabilization. It's much more difficult to achieve 4K quality that holds up on a movie theater screen, without visible artifacts, than to do an artistic music video.

https://youtu.be/eyAVWH61R8E?t=3m53s I would say Superman's quality didn't suffer for it. I would say cost is probably the most expensive part it's also just like "why bother", it's not CG, it's not "2d filming" so it's just niche, like the scenarios you would actually need this are very low.

That's interesting. 192 cameras is certainly expensive. Though they are doing 4DGS, with movement, so they have to capture every frame from different angles at the same time. I assume 3DGS for static environments (locations) would be a lot easier in terms of hardware. E.g. a single drone could collect photos for an hour and then they could create arbitrary simulated camera movements that couldn't be filmed conventionally. But again, the quality would have to be high in most cases. The nature of the Superman scene (some sort of hologram) is more forgiving, as it is inherently fake-looking, which helps excuse artifacts slipping through.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#232

Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?

1. Create a point cloud from a scene (either via lidar, or via photogrammetry from multiple images) 2. Replace each point of the point cloud with a fuzzy ellipsoid, that has a bunch of parameters for its position + size + orientation + view-dependent color (via spherical harmonics up to some low order) 3. If you render these ellipsoids using a differentiable renderer, then you can subtract the resulting image from th…

And what about the "mature enough" part? How has it changed / progressed recently?

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#233

I want to shoutout Nial Ashley (aka Llainwire) for doing this in 2023 as a solo act and doing the visuals himself as well - https://www.youtube.com/watch?v=M1ZXg5wVoUU A shame that kid was slept on. Allegedly (according to discord) he abandoned this because so many artists reached out to have him do this style of mv, instead of wanting to collaborate on music.

> so many artists reached out to have him do this style of mv, instead of wanting to collaborate on music Well yes, the visuals are awesome, while the music… isn’t.

I love HN because everyone is so different outside of the core purpose of the site. Sometimes people reference art, or a book or something, that I'd never would think to exist.

Llainwire was my top artist listens throughout 2023, so it’s always funny to bump into reactions that feel totally different from my world/my peers.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#234
post #20

Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?

Gaussian splatting is a way to record 3-dimensional video. You capture a scene from many angles simultaneously and then combine all of those into a single representation. Ideally, that representation is good enough that you can then, post-production, simulate camera angles you didn't originally record. For example, the camera orbits around the performers in this music video are difficult to imagine in real space. Eve…

Are meshes not used instead of gaussian splats only due to robustness reasons? I.e., if there were a piece of software that could reliably turn a colored point cloud into a textured mesh, would that be preferable?

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#235
post #52

Earlier quoted context omitted.

Similarly, the music video for Taylor Swif[0] (another track by A$AP Rocky) is just as surrealistic and weird in the best way possible, but with an eastern european flavor of it (which is obviously intentional and makes sense, given the filming location and being very on-the-nose with the theme). 0. https://youtu.be/5URefVYaJrA

Holy shit that’s great. I need to check a few more of his videos.

A$AP Rocky has always put out interesting videos, here are a few others that are pretty cool:

Yamborghini high: https://www.youtube.com/watch?v=tt7gP_IW-1w L$D: https://www.youtube.com/watch?v=Gx4JEBwVlXo

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#237

Earlier quoted context omitted.

Kinda, it all depends on platforms and eras. As a demoscener I see more of techdemo, indiedev and "game-mods" vibes in much of it (especially certain kinds of jankiness feel of visuals in some parts that demosceners doing more high end in general try to avoid but is more meme-worthy in game-mods/indiedev circles). Demos often aim for a bit more clean visuals (and palettes), blurriness is often tuned in another way. S…

> but sadly there's a tad of a stigma culturally as NN/"AI" methods has been fairly controversial within the scene The scene has always been a ridiculously conservative bunch. Back when 3dfx was new, using 3d acceleration was similarly controversial. The Pouet comments were scary similar to those today. All we need is a few demos that actually use these technologies with great results (instead of for laziness/slop),…

I consider the early years of 3d acceleration to be pretty bad days for the demoscene on the "high end" side.

Problem was that fixed pipelines were seriously limiting. Most of the cool effects of software rendering couldn't be done, and the lack of direct access to the hardware meant that you couldn't do many of the hardware tricks the demoscene is known for. It doesn't mean people couldn't be creative, but it was mostly limited to doing interesting 3D geometry and textures. Things started changing with the advent of shaders.

About AI, I think thing the demoscene is rather welcoming of "AI" as long as you use it creatively instead of doing what everyone else does (slop). In the topic of Gaussian splatting, look at the Revision 2025 invitation [1], there is a pretty cool scene featuring it, people loved that part.

[1] https://www.pouet.net/prod.php?which=103537

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#238

Too bad, but I managed to watch about 30 seconds of the video before getting motion sickness. Seems like a really cool technology, though. I wonder if anyone else got the same response, or it's just me.

Yes, just clicked on the video, instantly nauseous. I get motion sick generally.

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#239
post #20

Earlier quoted context omitted.

Gaussian splatting is a way to record 3-dimensional video. You capture a scene from many angles simultaneously and then combine all of those into a single representation. Ideally, that representation is good enough that you can then, post-production, simulate camera angles you didn't originally record. For example, the camera orbits around the performers in this music video are difficult to imagine in real space. Eve…

Are meshes not used instead of gaussian splats only due to robustness reasons? I.e., if there were a piece of software that could reliably turn a colored point cloud into a textured mesh, would that be preferable?

Not everything can be represented by textured meshes used in traditional photogrammetry (think Google Street View)

This includes sparse areas like fences, vegetation and the likes, but more importantly any material properties like reflections, specularity, opacity, etc.

Here's a few great examples: https://superspl.at/view?id=cf6ac78e

https://superspl.at/view?id=c67edb74

Re: Gaussian Splatting – A$AP Rocky "Helicopter" music video

#240

Earlier quoted context omitted.

https://developer.apple.com/av-foundation/ https://developer.apple.com/documentation/spatial/ Edit: As I'm digging, this seems to be focused on stereoscopic video as opposed to actual point clouds. It appears applications like cinematic mode use a monocular depth map, and their lidar outputs raw point cloud data.

A LIDAR point cloud from a single point of view is a mono-ocular depth map. Unless the LIDAR in question is like, using supernova level gamma rays or neutrino generators for the laser part to get density and albedo volumetric data for its whole distance range. You just can't see the back of a thing by knowing the shape of the front side with current technologies.

Right! My terminology may be imprecise here, but I believe there is still an important distinction:

The depth map stored for image processing is image metadata, meaning it calculates one depth per pixel from a single position in space. Note that it doesn't have the ability to measure that many depth values, so it measures what it can using LIDAR and focus information and estimates the rest.

On the other hand, a point cloud is not image data. It isn't necessarily taken from a single position, in theory the device could be moved around to capture addition angles, and the result is a sparse point cloud of depth measurements. Also, raw point cloud data doesn't necessarily come tagged with point metadata such as color.

I also note that these distinctions start to vanish when dealing with video or using more than one capture device.

Post reply on HN