Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeshGPT: Generating triangle meshes with decoder-only transformers

nihalsid.github.io

Post reply on HN
Print view View original

81–90 of 166 posts

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#81

Post by johnnyanmac »

Kaijo wrote: post #67
LarsDu88 wrote: post #47

As a machine learning engineer who dabbles with Blender and hobby gamedev, this is pretty impressive, but not quite to the point of being useful in any practical manner (as far as the limited furniture examples are concerned. A competent modeler can make these types of meshes in under 5 minutes, and you still need to seed the generation with polys. I imagine the next step will be to have the seed generation controlle…

The mesh topology here would see these rejected as assets for in basically any professional context. A competent modeler could make much higher quality models, more suited to texturing and deformation, in under five minutes. A speed modeler could make the same in under a minute. And a procedural system in something like Blender geonodes can already spit out an endless variety of such models. But the pace of progress…

I see it as a black triangle[0] more than anything else. Sounds like a really good first step that will scale to stuff that would take even a good modeler days to produce. That's where the real value will start to be seen.

[0]: https://rampantgames.com/blog/?p=7745

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#82

Post by GaggiX »

LarsDu88 wrote: post #47

As a machine learning engineer who dabbles with Blender and hobby gamedev, this is pretty impressive, but not quite to the point of being useful in any practical manner (as far as the limited furniture examples are concerned. A competent modeler can make these types of meshes in under 5 minutes, and you still need to seed the generation with polys. I imagine the next step will be to have the seed generation controlle…

A simple next step would be to simply scale the model, make it bigger, and train it on millions of images in the wild.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#83

Post by johnnyanmac »

andkenneth wrote: post #65
worldsayshi wrote: post #58

Earlier quoted context omitted.

I don't see that LLM's have come that much further in 3D animation than programming in this regard: It can spit out bits and pieces that looks okay in isolation but a human need to solve the puzzle. And often solving the puzzle means rewriting/redoing most of the pieces. We're safe for now but we should learn how to leverage the new tech.

This is the "your job won't be taken away by AI, it will be taken away by someone who knows how to leverage AI better than you"

Probably, but isn't that how most if the technical fields go? Software in particular moves blazing fast and you need to adapt to the market quickly to be marketable.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#84

Post by topwalktown »

enjeyw wrote: post #52
_hark wrote: post #45

Earlier quoted context omitted.

NNs are typically continuous/differentiable so you can do gradient-based learning on them. We often want to use some of the structure the NN has learned to represent data efficiently. E.g., we might take a pre-trained GPT-type model, and put a passage of text through it, and instead of getting the next-token prediction probability (which GPT was trained on), we just get a snapshot of some of the activations at some i…

If you’re willing, I’d love your insight on the “why one might want to do this”. Conceptually I understand embedding quantization, and I have some hint of why it works for things like WAV2VEC - human phonemes are (somewhat) finite so forcing the representation to be finite makes sense - but I feel like there’s a level of detail that I’m missing regarding whats really going on and when quantisation helps/harms that I…

Quantization also works as regularization; it stops the neural network from being able to use arbitrarily complex internal rules.

But really it's only really useful if you absolutely need to have a discrete embedding space for some sort of downstream usage. VQVAEs can be difficult to get to converge, they have problems stemming from the approximation of the gradient like codebook collapse

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#85

Post by [deleted] »

LarsDu88 wrote: post #47

As a machine learning engineer who dabbles with Blender and hobby gamedev, this is pretty impressive, but not quite to the point of being useful in any practical manner (as far as the limited furniture examples are concerned. A competent modeler can make these types of meshes in under 5 minutes, and you still need to seed the generation with polys. I imagine the next step will be to have the seed generation controlle…

[deleted]

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#86

Post by GaggiX »

toxik wrote: post #27

This was done years ago, with transformers. It was then dubbed Polygen.

First, you use the word "transformers" to mean "autoregressive models", they are not synonymous, second, this model beats Polygen on every metric, it's not even close.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#87

Post by hipadev23 »

johnnyanmac wrote: post #79
empath-nirvana wrote: post #63

Earlier quoted context omitted.

> A competent modeler can make these types of meshes in under 5 minutes. I don't think this general complaint about AI workflows is that useful. Most people are not a competent . Most people don't know a competent or can't afford to hire one. Even something that takes longer than a professional do at worse quality for many things is better than _nothing_ which is the realistic alternative for most people who would us…

>Most people don't know a competent or can't afford to hire one May be relevant in the long run, but it'll probably be 5+ years before this is commercially available. And it won't be cheap either, so out of the range of said people who can't hire a competent That's why a lot of this stuff is pitched to companies with competent people instead of offered as a general product to download.

> but it'll probably be 5+ years before this is commercially available

I think you should look at the progress of image, text, and video generation over the past 12 months and re-asses your timeline prediction.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#88

Post by hipadev23 »

LarsDu88 wrote: post #47

As a machine learning engineer who dabbles with Blender and hobby gamedev, this is pretty impressive, but not quite to the point of being useful in any practical manner (as far as the limited furniture examples are concerned. A competent modeler can make these types of meshes in under 5 minutes, and you still need to seed the generation with polys. I imagine the next step will be to have the seed generation controlle…

> A competent modeler can make these types of meshes in under 5 minutes

Sweet. Can you point me to these modelers who work on-demand and bill for their time in 5 minute increments? I’d love to be able to just pay $1-2 per model and get custom dropped into my game when I need it.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#89

Post by mattigames »

valine wrote: post #26

Even if this is “only” mesh autocomplete, it is still massively useful for 3D artists. There’s a disconnect right now between how characters are sculpted and how characters are animated. You’d typically need a time consuming step to retopologize your model. Transformer based retopology that takes a rough mesh and gives you clean topology would be a big time saver. Another application: take the output of your gaussian…

Lol for 3D artists, this will be used 99% by people who have have never created a mesh by hand in their lifes; to replace their need to hire a 3D artist: programmers who don't want (or can't) pay a designer, architects who never learned nothing other than CAD, fiver "jobs", et al

I don't think people here realize how are we inching to automating the automation itself, and the programmers who will be able to make a living out of this will be a tiny fraction of those who can make a living out of it today.

Post reply on HN