Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeshGPT: Generating triangle meshes with decoder-only transformers

nihalsid.github.io

Post reply on HN
Print view View original

121–130 of 166 posts

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#121

Post by johnnyanmac »

jamilton wrote: post #95
johnnyanmac wrote: post #79

Earlier quoted context omitted.

>Most people don't know a competent or can't afford to hire one May be relevant in the long run, but it'll probably be 5+ years before this is commercially available. And it won't be cheap either, so out of the range of said people who can't hire a competent That's why a lot of this stuff is pitched to companies with competent people instead of offered as a general product to download.

Is there a reason to expect it'd be significantly more expensive than current-gen LLM? Reading the "Implementation Details" section, this was done with GPT2-medium, and assuming running it is about as intensive as the original GPT2, it can be run (slowly) on a regular computer, without a graphics card. Seems reasonable to assume future versions will be around GPT-3/4's price.

Perhaps not, but it begs the question of if GPT is affordable for a dev to begin with. I don't know how they would monetize this sort of work so it's hard to say. But making game models probably requires a lot more processing power than generating text or static images.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#122

Post by quickthrower2 »

dvngnt_ wrote: post #110
hipadev23 wrote: post #88

Earlier quoted context omitted.

> A competent modeler can make these types of meshes in under 5 minutes Sweet. Can you point me to these modelers who work on-demand and bill for their time in 5 minute increments? I’d love to be able to just pay $1-2 per model and get custom dropped into my game when I need it.

they said competent though no cheap

but the AI will be cheap. $1 per model would be the OpenAI wrapper’s price. Let alone the wholesale price.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#123

Post by quickthrower2 »

chongli wrote: post #19
stuckinhell wrote: post #8

Earlier quoted context omitted.

I think indie game development is dead with these techniques. Instead big companies will create "make your own game" games. Indie games already seems pretty derivative these days. I think this tech will kill them in mid-term as big companies use them.

“Make your own game” games will never replace regular games. They target totally different interests. People who play games (vast majority) just want to play an experience created by someone else. People who like “make your own game” games are creative types who just use that as a jumping off point to becoming a game designer. It’s no different than saying “these home kitchen appliances are really gonna kill off the…

But the make your own game games are saleable games. Or at least they will be with enough AI. The Roblox minecraft for example is super realistic.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#124

Post by fjkdlsjflkds »

ganzuul wrote: post #77
shaileshm wrote: post #4

This is what a truly revolutionary idea looks like. There are so many details in the paper. Also, we know that transformers can scale. Pretty sure this idea will be used by a lot of companies to train the general 3D asset creation pipeline. This is just too great. "We first learn a vocabulary of latent quantized embeddings, using graph convolutions, which inform these embeddings of the local mesh geometry and topolog…

...Is graph convolution matrix factorization by another name?

No... a graph convolution is just a convolution (over a graph, like all convolutions).

The difference from a "normal" convolution is that you can consider arbitrary connectivity of the graph (rather than the usual connectivity induced by a regular Euclidian grid), but the underlying idea is the same: to calculate the result of the operation at any single place (i.e., node), you need to perform a linear operation over that place (i.e., node) and its neighbourhood (i.e., connected nodes), the same way that (e.g.) in a convolutional neural network, you calculate the value of a pixel by considering its value and that of its neighbours, when performing a convolution.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#125

Post by BrokrnAlgorithm »

I'm not a 3D artist, but why are we still, for lack of a better word, "stuck" with having / wanting to use simple meshes? I appreciate the simplicity, but isn't this an unnecessary limitation of mesh generation? It feels like an approach that imitates the constraints of having both limited hardware and artist resources. Shouldn't AI models help us break these boundaries?

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#126

Post by ipsum2 »

BrokrnAlgorithm wrote: post #125

I'm not a 3D artist, but why are we still, for lack of a better word, "stuck" with having / wanting to use simple meshes? I appreciate the simplicity, but isn't this an unnecessary limitation of mesh generation? It feels like an approach that imitates the constraints of having both limited hardware and artist resources. Shouldn't AI models help us break these boundaries?

We're not stuck on meshes. Check out neural radiance fields as an alternative.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#127

Post by LarsDu88 »

hipadev23 wrote: post #87
johnnyanmac wrote: post #79

Earlier quoted context omitted.

>Most people don't know a competent or can't afford to hire one May be relevant in the long run, but it'll probably be 5+ years before this is commercially available. And it won't be cheap either, so out of the range of said people who can't hire a competent That's why a lot of this stuff is pitched to companies with competent people instead of offered as a general product to download.

> but it'll probably be 5+ years before this is commercially available I think you should look at the progress of image, text, and video generation over the past 12 months and re-asses your timeline prediction.

I have no doubt that 3d modeling will become commodified in the same way that art has with the dawn of AI art generation over the past year.

I honestly think we'll get there within 18 months.

My skepticism is whether the technique described here will be the basis of what people will be using in ~2 years to replace their low level static 3d asset generation.

There are several techniques out there, leveraging different sources of data right now. This looks like a step in the right direction, but who knows.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#128

Post by fireant »

ipsum2 wrote: post #126
BrokrnAlgorithm wrote: post #125

I'm not a 3D artist, but why are we still, for lack of a better word, "stuck" with having / wanting to use simple meshes? I appreciate the simplicity, but isn't this an unnecessary limitation of mesh generation? It feels like an approach that imitates the constraints of having both limited hardware and artist resources. Shouldn't AI models help us break these boundaries?

We're not stuck on meshes. Check out neural radiance fields as an alternative.

My understanding is that it's quite hard to make convex objects with radiance fields, right? For example the furniture in OP would be quite problematic.

We can create radiance fields with photogrammetry, but IMO we need much better algorithms for transforming these into high quality triangle meshes that are usable in lower triangle budget media like games.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#129

Post by BrokrnAlgorithm »

fireant wrote: post #128
ipsum2 wrote: post #126

Earlier quoted context omitted.

We're not stuck on meshes. Check out neural radiance fields as an alternative.

My understanding is that it's quite hard to make convex objects with radiance fields, right? For example the furniture in OP would be quite problematic. We can create radiance fields with photogrammetry, but IMO we need much better algorithms for transforming these into high quality triangle meshes that are usable in lower triangle budget media like games.

"Lower triangle budget media" is what I wonder if its still a valid problem. Modern game engines coupled with modern hardware can already render insane number of triangles. It feels like the problem is rather in engines not handling LOD correctly (see city skylines 2), although stuff like UE5 nanite seems to have taken the right path here.

I suppose though there is a case for AI models for example doing what nanite does entirely algorithmically and research like this paper may come in handy there.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#130

Post by BrokrnAlgorithm »

ipsum2 wrote: post #126
BrokrnAlgorithm wrote: post #125

I'm not a 3D artist, but why are we still, for lack of a better word, "stuck" with having / wanting to use simple meshes? I appreciate the simplicity, but isn't this an unnecessary limitation of mesh generation? It feels like an approach that imitates the constraints of having both limited hardware and artist resources. Shouldn't AI models help us break these boundaries?

We're not stuck on meshes. Check out neural radiance fields as an alternative.

I was referring to being stuck with having to create simple / low tri polygonal meshes as opposed to using complex poly meshes such as photogrammetry would provide. The paper specifically addresses clean low poly meshes as opposed to what they call complex iso surfaces created by photogrammetry and other methods
Post reply on HN