Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Better Gaussian Splatting in Julia

pxl-th.github.io

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Better Gaussian Splatting in Julia

#21

Post by jaccola »

ghostly_s wrote: post #20
yorwba wrote: post #10

Earlier quoted context omitted.

Those are the areas that are occluded in the input images. If you look at the bushes, you can see that the front is clean while the back is a mess. Probably the starting position could be chosen a bit better.

This raises the question, do any of these projects have workflows for combining images taken from different positions? Or does that just work out of the box? (I have no machines atm with anything better than 10yo eGPUs ;~<)

Yes the whole field could more or less be described as combining images taken from different positions :)

Re: Better Gaussian Splatting in Julia

#22

Post by Cieric »

ghostly_s wrote: post #19
Cieric wrote: post #13

I'm still very fascinated by gaussian splatting, but the issue I always run into is the need to use colmap to generate the camera positions and get the initial point cloud. Which colmap effectively still needs Nvidia for dense point clouds last I checked. I'm curious though does anyone have anything else they've tried that worked well? I've tried a few different things (most of the time getting stuck on just compilin…

TFA mentions this project has backends for : AMD GPU (AMDGPU.jl) NVIDIA GPU (CUDA.jl) MacBook GPU(Metal.jl)

I'm sorry, but the article doesn't cover anything about the preprocessing steps that are required for 90% of gaussian splat implementations. That preprocessing step is almost always colmap which takes pictures or videos and uses SfM to figure out the camera parameters and their position in the 3d scene, it then does basic point reconstruction and dense point reconstruction. That info is what gaussian splats take in, it converts the points from a colmap project into gaussians and then optimizes them through differentiable rendering using backprop to optimize the gaussians to better match the images at the calculated positions of the camera from colmap. Looking over the project files also shows that they consume colmap projects, my question wasn't about the gaussian splat implementation, it was about the defacto standard colmap preprocessing step that is also required.

Re: Better Gaussian Splatting in Julia

#23

Post by [deleted] »

corysama wrote: post #17
Cieric wrote: post #13

I'm still very fascinated by gaussian splatting, but the issue I always run into is the need to use colmap to generate the camera positions and get the initial point cloud. Which colmap effectively still needs Nvidia for dense point clouds last I checked. I'm curious though does anyone have anything else they've tried that worked well? I've tried a few different things (most of the time getting stuck on just compilin…

I know there have been papers and research projects about GS without COLMAP and GS from videos. But, I don't have links handy. Best I can do is link you to https://x.com/RadianceFields and https://radiancefields.com/ They have all the news about GS every day.

[deleted]

Re: Better Gaussian Splatting in Julia

#26

Post by yorwba »

ghostly_s wrote: post #20
yorwba wrote: post #10

Earlier quoted context omitted.

Those are the areas that are occluded in the input images. If you look at the bushes, you can see that the front is clean while the back is a mess. Probably the starting position could be chosen a bit better.

This raises the question, do any of these projects have workflows for combining images taken from different positions? Or does that just work out of the box? (I have no machines atm with anything better than 10yo eGPUs ;~<)

The article includes a visualization of the different positions that images were taken from: https://pxl-th.github.io/video/camera-frustum.mp4 It's just that they're all bunched around the statue and the starting position for the rendering is a bit farther away, where there's direct line of sight to spots not covered by those images.

Re: Better Gaussian Splatting in Julia

#27

Post by fragmede »

Cieric wrote: post #22
ghostly_s wrote: post #19

Earlier quoted context omitted.

TFA mentions this project has backends for : AMD GPU (AMDGPU.jl) NVIDIA GPU (CUDA.jl) MacBook GPU(Metal.jl)

I'm sorry, but the article doesn't cover anything about the preprocessing steps that are required for 90% of gaussian splat implementations. That preprocessing step is almost always colmap which takes pictures or videos and uses SfM to figure out the camera parameters and their position in the 3d scene, it then does basic point reconstruction and dense point reconstruction. That info is what gaussian splats take in,…

What hardware do you have? I've been using colmap on my MacBook. It's been working alright.

Re: Better Gaussian Splatting in Julia

#28

Post by ghostly_s »

jaccola wrote: post #21
ghostly_s wrote: post #20

Earlier quoted context omitted.

This raises the question, do any of these projects have workflows for combining images taken from different positions? Or does that just work out of the box? (I have no machines atm with anything better than 10yo eGPUs ;~<)

Yes the whole field could more or less be described as combining images taken from different positions :)

As I understand the typical workflow starts with a stack of frames which have a small offset in position /orientation which the model is able to derive due to sufficient similarities to the adjacent frames; if I created three stacks like that which are internally self-similar but were taken from three viewpoints 20 meters apart, so perhaps only a small portion of the scene is common between them, could the model still orient them into a single scene? I suppose you could just capture the "transition" frames as you move your camera from one viewpoint to the next as well and then it would be treated as a single path?

Re: Better Gaussian Splatting in Julia

#29

Post by Cieric »

fragmede wrote: post #27
Cieric wrote: post #22

Earlier quoted context omitted.

I'm sorry, but the article doesn't cover anything about the preprocessing steps that are required for 90% of gaussian splat implementations. That preprocessing step is almost always colmap which takes pictures or videos and uses SfM to figure out the camera parameters and their position in the 3d scene, it then does basic point reconstruction and dense point reconstruction. That info is what gaussian splats take in,…

What hardware do you have? I've been using colmap on my MacBook. It's been working alright.

It's been a while since I've tried to run it, but it's a mix of Nvidia and AMD GPUs and only AMD CPUs (everything portable is all AMD though and that's where my gripes are.) I'm sure the opencl backend has gotten better over time, but it always seems to drag behind the cuda backend. At this point I wish we could just have a vulkan backend and be done with it.

Thanks for bringing that up though, after a quick search that I guess I haven't done in a long time, it appears that colmap does now have a hip backend. So I guess that's going to be my weekend now.

Re: Better Gaussian Splatting in Julia

#30

Post by ChrisClark »

chrisjj wrote: post #16
ChrisClark wrote: post #15

Earlier quoted context omitted.

By not taking photos of it... what else? The data wasn't there, so it couldn't be generated Actually, this might be a lack of understanding, look up what gaussian splatting is.

> By not taking photos of it... That's not occlusion.

lol, there can be more than one explanation, you're being purposely obtuse.

Occlusion can block you from taking a photo.

Not taking a photo can block you from taking a photo.

Sometimes, there are two things that happen!

Seriously, you'd understand if you actually bothered to look up what Gaussian Splatting is

Post reply on HN