Bolt3D: Generating 3D Scenes in Seconds

Games are the least of it, the vast majority of scientific applications to do with physics use meshes rather than point clouds.

This is because a point cloud does not represent a surface or a volume until the points are connected to form, well, a surface or a volume.

And physical problems are most often defined over surfaces or volumes. For instance, waves don't propagate over sparse sets of points, but within continuous domains.

However, for applications where geometric accuracy is needed, I think you wouldn't want to use a method based on a minimal number of photographs anyways. For instance, the Lascaux cavern was mapped in 3D a decade ago based on "good old" algorithms (not machine learning) and instruments (more sophisticated than a phone camera). So these critiques are missing the point, in my opinion. These Gaussian Splatting methods are very impressive for the constraints they operate under!

What's the use case for putting AI into everything? Pretty much every AI product so far has been and still is subject to hallucinations and inaccuracies and on top of that it's hugely computing intensive. Sure, it's the best we have right now and it allows us to do things that were previously next to impossible with manual programming work, but it's far from being something that's actually viable. And what would be the use case for turning a picture into an approximated 3d mesh that is only really complete from one angle? LIDAR does a stunningly accurate job at that already, reproducibly (although granted that this cannot retroactively be applied to existing photos).

So I agree with you, but to be fair it is neat, and I think academia should be allowed to try things with little to no *immediate* commercial value. Being "neat" is enough IMO if there's enough resources to go around.

In the long run, yeah this *exact* application is sort of pointless. I expected to see the lens parameters factored into the process. It's not. This would mean that everything is not only dimensionally inaccurate since there's no reference measurement, but also proportionally inaccurate to other things in the scene. You can actually see the effect of that on the "flower car" example. (the entire shape of the car is warped) Let alone the fact that the entire scene that can't be seen in the original photo is made up.

Maybe someone would use this to make game assets? But you'd need to fix them up a ton before using them. Other sibling comments make the point that there's no wireframes... so we can assume the polygon count here is insane.

Either way... it's just neat.

I don't know what you mean by lacking structure, but perhaps you are not aware of all the tools that exist, because fixing surface meshes is a rather classic problem. Just type "surface remeshing" or "surface mesh optimization" on google scholar and you'll see thousands of results.

This is a separate problem from triangulation (turning point clouds into meshes) done with entirely different algorithms. It's likely the software you used for this assumes the user will then turn to other software to improve their surface mesh.

Even for operations that are naturally in sequence, you will often find the software to carry out those steps is separated. For instance turning CAD into a surface mesh is one software, turning a surface mesh into a tetrahedral volume mesh another (if those are hexahedra, then yet another), and then optimizing or adapting those meshes is done by yet another piece of software. And yet these steps are carried out each time an engineer goes from CAD to physical simulation. So it's entirely possible the triangulation software you used does not implement any kind of surface optimization and assumes the user will then find something else to deal with that.

Apples, not oranges. This isn't for 3d models and scenes- think of it as a fancy version of streetview or an apartment walkthrough. Those project pictures onto a sphere around you. Gaussian splats are an improvement that uses multiple images to interpolate viewpoints- take two pictures from different angles, and guassian splats let you take a look from between those views, or above or below them.

This method uses ai to generate even more unseen structure, so with relatively few images you can still represent a real scene with some level of fidelity. It will never need dynamic lights or animation because the point is just to look as close as possible to a still image. Splats do that FAR better and more efficiently than you ever could with dynamic lighting, triangulated models, and visual effects.