MeshGPT: Generating triangle meshes with decoder-only transformers

This looks really cool! Seems like it would be an incredible boon for an indie game developer to generate a large pool of assets!

This is what a truly revolutionary idea looks like. There are so many details in the paper. Also, we know that transformers can scale. Pretty sure this idea will be used by a lot of companies to train the general 3D asset creation pipeline. This is just too great.

"We first learn a vocabulary of latent quantized embeddings, using graph convolutions, which inform these embeddings of the local mesh geometry and topology. These embeddings are sequenced and decoded into triangles by a decoder, ensuring that they can effectively reconstruct the mesh."

This idea is simply beautiful and so obvious in hindsight.

"To define the tokens to generate, we consider a practical approach to represent a mesh M for autoregressive generation: a sequence of triangles."

More from paper. Just so cool!

On my phone so I’ve only read this promo page - could this approach be modified for surface reconstruction from a 3D point cloud?

I think indie game development is dead with these techniques. Instead big companies will create "make your own game" games.

Indie games already seems pretty derivative these days. I think this tech will kill them in mid-term as big companies use them.