Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeshGPT: Generating triangle meshes with decoder-only transformers

nihalsid.github.io

Post reply on HN
Print view View original

21–30 of 166 posts

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#21

Post by tayo42 »

CamperBob2 wrote: post #11
sram1337 wrote: post #5

What is the input? Is it converting a text query like "chair" to a mesh? edit: Seems like mesh completion is the main input-output method, not just a neat feature.

That's what I was wondering. From the diagram it looks like the input is other chair meshes, which makes it somewhat less interesting.

Really the hardest thing with art is details and usually seperates good from bad. So if you can sketch what you want roughly without skill and have the details generated, that's extremely useful. And image to image with the existing diffusion models is useful and popular.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#23

Post by nextworddev »

kranke155 wrote: post #7

My chosen profession (3D / filmmaking) feels like being in some kind of combat trench at the moment. Both fascinating and scary

What do you ascertain the use case of this in your field? Does it seem high quality? (I have no context)

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#24

Post by treyd »

CamperBob2 wrote: post #11
sram1337 wrote: post #5

What is the input? Is it converting a text query like "chair" to a mesh? edit: Seems like mesh completion is the main input-output method, not just a neat feature.

That's what I was wondering. From the diagram it looks like the input is other chair meshes, which makes it somewhat less interesting.

There's also examples of tables, lamps, couches, etc in the video.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#25

Post by dexwiz »

stuckinhell wrote: post #8
chongli wrote: post #2

This looks really cool! Seems like it would be an incredible boon for an indie game developer to generate a large pool of assets!

I think indie game development is dead with these techniques. Instead big companies will create "make your own game" games. Indie games already seems pretty derivative these days. I think this tech will kill them in mid-term as big companies use them.

The platform layer of the "make your own game" game is always too heavy and too limited to compete with a dedicated engine in the long run. Also the monetization strategy is bad for professionals.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#26

Post by valine »

Even if this is “only” mesh autocomplete, it is still massively useful for 3D artists. There’s a disconnect right now between how characters are sculpted and how characters are animated. You’d typically need a time consuming step to retopologize your model. Transformer based retopology that takes a rough mesh and gives you clean topology would be a big time saver.

Another application: take the output of your gaussian splatter or diffusion model and run it through MeshGPT. Instant usable assets with clean topology from text.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#28

Post by mlsu »

The next breakthrough will be the UX to create 3d scenes in front of a model like this, in VR. This would basically let you _generate_ a permanent, arbitrary 3D environment, for any environment for which we have training data.

Diffusion models could be used to generate textures.

Mark is right and so so early.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#29

Post by toxik »

valine wrote: post #26

Even if this is “only” mesh autocomplete, it is still massively useful for 3D artists. There’s a disconnect right now between how characters are sculpted and how characters are animated. You’d typically need a time consuming step to retopologize your model. Transformer based retopology that takes a rough mesh and gives you clean topology would be a big time saver. Another application: take the output of your gaussian…

What you have to understand is that these methods are very sensitive to what is in distribution and out of distribution. If you just plug in user data, it will likely not work.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#30

Post by [deleted] »

owenpalmer wrote: post #17
all2 wrote: post #14

Earlier quoted context omitted.

You prompt this LLM using 3D meshes for it to complete, in the same manner you use language to prompt language specific LLMs.

That's what it seems like. Although this is not an LLM. > Inspired by recent advances in powerful large language models, we adopt a sequence-based approach to autoregressively generate triangle meshes as sequences of triangles. It's only inspired by LLMs

[deleted]
Post reply on HN