Painting with Gaussians
11–20 of 24 posts
Re: Painting with Gaussians
#12Re: Painting with Gaussians
#13I investigated a gradient-descent-based concept adjacent to this back in like 2023 [0] inspired by the original 3DGS paper, I was pretty happy with the results I got with back-to-front alpha-blending and polar-coordinate gaussians for variable-curvature brush strokes. One of these days I should add another example gif to the repo besides just the Mona Lisa one that's currently there, but that's the only one I've got…
Re: Painting with Gaussians
#14The effect looks decent in the foreground of some pictures, particularly the wolf and the cat. It really exaggerates the depth of field in the background, though, and those areas feel like someone ran a posterise after a blur, rather than brush strokes. Might just be a side effect of the author's predilection for bokeh, I guess. The Tokyo shot looks good throughout.
Re: Painting with Gaussians
#15I investigated a gradient-descent-based concept adjacent to this back in like 2023 [0] inspired by the original 3DGS paper, I was pretty happy with the results I got with back-to-front alpha-blending and polar-coordinate gaussians for variable-curvature brush strokes. One of these days I should add another example gif to the repo besides just the Mona Lisa one that's currently there, but that's the only one I've got…
It's like… the way a trained human would paint feels kind of like how MSB radix sort makes an array look more ordered, going from crude to refined. The way your algorithm builds up an i|age feels like I'm looking at a radix sort that alternates between a stable MSB and LSB radix sort, which is weird as heck. It's fascinating, really.
Re: Painting with Gaussians
#16I investigated a gradient-descent-based concept adjacent to this back in like 2023 [0] inspired by the original 3DGS paper, I was pretty happy with the results I got with back-to-front alpha-blending and polar-coordinate gaussians for variable-curvature brush strokes. One of these days I should add another example gif to the repo besides just the Mona Lisa one that's currently there, but that's the only one I've got…
Re: Painting with Gaussians
#17Re: Painting with Gaussians
#18So interestingly, one distinguishing difference between oil paintings and acrylic paintings is that the former has more translucency, whereas the latter can be more easily layered on in a way that covers up the lower layers. This is what, according to one of my mentors in art school who was a painter, allows oil paintings to copy the feeling of real skin so well: it allows a painter who mastered the medium to create a similar subsurface scattering effect. That's also another reason why paintings in real life can look so different than images on a screen, aside from the much greater gamut than even the best color screens we have and the actual surface texture of the paint strokes. The downside is that any mistakes made on deeper layers will come through in higher layers. Especially if you paint using the techniques used by the old masters.
Anyway, this suggests your gaussian splats essentially are closer to "oil painting"-like. Makes me wonder if you could also encode this subsurface scattering and texture somehow, so that viewing the splats in 3D would recreate some of that missing effect of looking at a real painting!
Also, on the other end of the spectrum: do you think the algorithm could be modified to create SVG output? Then it might be useful as an artistic "upscaling" filter too.
(Also, speaking as someone with protanomaly, I'm guessing the links have some kind of turquise or light teal color to them, but it is effectively bright gray to me. And since they have almost the same brightness perception as the white text I genuinely cannot tell where the links are (except when the text calls them out). Since this is likely to affect anywhere between 5% to 10% of all men visiting your blog that might be worth changing.)
Re: Painting with Gaussians
#19How long is the frame time on the GPU implementation? Is there any hope for something like this to run at 60fps as a post process for a game? Can the data from a 3D scene be used to skip some of the initial analysis steps?
Re: Painting with Gaussians
#20The effect looks decent in the foreground of some pictures, particularly the wolf and the cat. It really exaggerates the depth of field in the background, though, and those areas feel like someone ran a posterise after a blur, rather than brush strokes. Might just be a side effect of the author's predilection for bokeh, I guess. The Tokyo shot looks good throughout.
It was more of a stylistic choice on my part. I could add more layers to flesh out background details. And it's Hong Kong actually. :)