Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeshGPT: Generating triangle meshes with decoder-only transformers

nihalsid.github.io

Post reply on HN
Print view View original

51–60 of 166 posts

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#51

Post by th0ma5 »

LarsDu88 wrote: post #47

As a machine learning engineer who dabbles with Blender and hobby gamedev, this is pretty impressive, but not quite to the point of being useful in any practical manner (as far as the limited furniture examples are concerned. A competent modeler can make these types of meshes in under 5 minutes, and you still need to seed the generation with polys. I imagine the next step will be to have the seed generation controlle…

This is a very underrated comment... As with any tech demo, I'd they don't show it, it can't do it. It is very very easy to imagine a generalization of these things to other purposes, which, if it could do it, would be a different presentation.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#52

Post by enjeyw »

_hark wrote: post #45
tomcam wrote: post #41

Earlier quoted context omitted.

Can someone explain quantized embeddings to me?

NNs are typically continuous/differentiable so you can do gradient-based learning on them. We often want to use some of the structure the NN has learned to represent data efficiently. E.g., we might take a pre-trained GPT-type model, and put a passage of text through it, and instead of getting the next-token prediction probability (which GPT was trained on), we just get a snapshot of some of the activations at some i…

If you’re willing, I’d love your insight on the “why one might want to do this”.

Conceptually I understand embedding quantization, and I have some hint of why it works for things like WAV2VEC - human phonemes are (somewhat) finite so forcing the representation to be finite makes sense - but I feel like there’s a level of detail that I’m missing regarding whats really going on and when quantisation helps/harms that I haven’t been able to gleam from papers.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#53

Post by angra_mainyu »

stuckinhell wrote: post #8
chongli wrote: post #2

This looks really cool! Seems like it would be an incredible boon for an indie game developer to generate a large pool of assets!

I think indie game development is dead with these techniques. Instead big companies will create "make your own game" games. Indie games already seems pretty derivative these days. I think this tech will kill them in mid-term as big companies use them.

I couldn't disagree more. RPGMaker didn't kill RPGs, Unity/Godot/Unreal didn't kill games, Minecraft didn't kill games, and Renpy didn't kill VNs.

Far more people prefer playing games than making them.

We'll probably see a new boom of indie games instead. Don't forget, a large part of what makes the gaming experience unique is the narrative elements, gameplay, and aesthetics - none of which are easily replaceable.

This empowers indie studios to hit a faster pace on one of the most painful areas of indie game dev: asset generation (or at least for me as a solo dev hobbyist).

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#54

Post by stuckinhell »

angra_mainyu wrote: post #53
stuckinhell wrote: post #8

Earlier quoted context omitted.

I think indie game development is dead with these techniques. Instead big companies will create "make your own game" games. Indie games already seems pretty derivative these days. I think this tech will kill them in mid-term as big companies use them.

I couldn't disagree more. RPGMaker didn't kill RPGs, Unity/Godot/Unreal didn't kill games, Minecraft didn't kill games, and Renpy didn't kill VNs. Far more people prefer playing games than making them. We'll probably see a new boom of indie games instead. Don't forget, a large part of what makes the gaming experience unique is the narrative elements, gameplay, and aesthetics - none of which are easily replaceable. Th…

Sorry I guess I wasn't clear. None of those things made games automatically. The future is buying a game making game, and saying I want a zelda clone but funnier.

The ai game framework handles the full game creation pipeline.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#55

Post by stuckinhell »

chongli wrote: post #19
stuckinhell wrote: post #8

Earlier quoted context omitted.

I think indie game development is dead with these techniques. Instead big companies will create "make your own game" games. Indie games already seems pretty derivative these days. I think this tech will kill them in mid-term as big companies use them.

“Make your own game” games will never replace regular games. They target totally different interests. People who play games (vast majority) just want to play an experience created by someone else. People who like “make your own game” games are creative types who just use that as a jumping off point to becoming a game designer. It’s no different than saying “these home kitchen appliances are really gonna kill off the…

Hmm I think it will destroy the market in a couple ways.

AI creating video games would drastically increase the volume of games available in the market. This surge in supply could make it harder for indie games to stand out, especially if AI-generated games are of high quality or novelty. It could also lead to even more indie saturation( the average indie makes less than 1000 dollars).

As the market expectations shift, I think most indie development dies unless you are already rich or basically have patronage from rich clients.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#56

Post by visarga »

enjeyw wrote: post #52
_hark wrote: post #45

Earlier quoted context omitted.

NNs are typically continuous/differentiable so you can do gradient-based learning on them. We often want to use some of the structure the NN has learned to represent data efficiently. E.g., we might take a pre-trained GPT-type model, and put a passage of text through it, and instead of getting the next-token prediction probability (which GPT was trained on), we just get a snapshot of some of the activations at some i…

If you’re willing, I’d love your insight on the “why one might want to do this”. Conceptually I understand embedding quantization, and I have some hint of why it works for things like WAV2VEC - human phonemes are (somewhat) finite so forcing the representation to be finite makes sense - but I feel like there’s a level of detail that I’m missing regarding whats really going on and when quantisation helps/harms that I…

Maybe it helps to point out that the first version of Dall-E (of 'baby daikon radish in a tutu walking a dog' fame) used the same trick, but they quantized the image patches.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#57

Post by stuckinhell »

CamperBob2 wrote: post #13
stuckinhell wrote: post #8

Earlier quoted context omitted.

I think indie game development is dead with these techniques. Instead big companies will create "make your own game" games. Indie games already seems pretty derivative these days. I think this tech will kill them in mid-term as big companies use them.

For values of "dead" equal to "Now people who aren't 3D artists and can't afford to hire them will be able to make games," maybe. User name checks out.

AI is already taking video game illustrators’ jobs in China https://restofworld.org/2023/ai-image-china-video-game-layof...

It feels like a countdown until every creative in the videogame industry is automated.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#58

Post by worldsayshi »

kranke155 wrote: post #7

My chosen profession (3D / filmmaking) feels like being in some kind of combat trench at the moment. Both fascinating and scary

I don't see that LLM's have come that much further in 3D animation than programming in this regard: It can spit out bits and pieces that looks okay in isolation but a human need to solve the puzzle. And often solving the puzzle means rewriting/redoing most of the pieces.

We're safe for now but we should learn how to leverage the new tech.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#59

Post by toxik »

dymk wrote: post #38
toxik wrote: post #34

Earlier quoted context omitted.

I read the article. It has exactly the same limitations as Polygen from what I can tell.

Their comparison against PolyGen looks like it's a big improvement. What are the limitations that this has in common with PolyGen that make it still not useful?

I don’t think it’s as widely applicable as they try to make it seem. I have worked specifically with PolyGen, and the main problem is “out of distribution” data. Basically anything you want to do will likely be outside the training distribution. This surfaces as sequencing. How do you determine which triangle or vertex to place first? Why would a user do it that way? What if I want to draw a table with the legs last? Cannot be done. The model is autoregressive.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#60

Post by ShamelessC »

owenpalmer wrote: post #17
all2 wrote: post #14

Earlier quoted context omitted.

You prompt this LLM using 3D meshes for it to complete, in the same manner you use language to prompt language specific LLMs.

That's what it seems like. Although this is not an LLM. > Inspired by recent advances in powerful large language models, we adopt a sequence-based approach to autoregressively generate triangle meshes as sequences of triangles. It's only inspired by LLMs

The only difference is the label, really. The underlying transformer architecture and the approach of using a codebook is identical to a large language model. The same approach was also used originally for image generation in DALL-E 1.
Post reply on HN