Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeshGPT: Generating triangle meshes with decoder-only transformers

nihalsid.github.io

Post reply on HN
Print view View original

111–120 of 166 posts

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#111

Post by kranke155 »

orbital-decay wrote: post #74
kranke155 wrote: post #7

My chosen profession (3D / filmmaking) feels like being in some kind of combat trench at the moment. Both fascinating and scary

I don't know, 3D CGI has already been moving at the breakneck speed for the last three decades without any AI. Today's tools are qualitatively different (sculpting, simulation, auto-rigging etc etc etc).

3D CGI has gotten faster, but I haven’t seen any qualitative jump for quite some time.

IMO the last time a major tech advance was visible was Davy Jones on the Pirates films. That was a fully photorealistic animated character that was plausible as a hero character in a major feature. That was a breakthrough. After that a lot of refinement and speeding up.

This is different. I have some positivity about it, but it’s getting hard to keep track of everything that’s going on tbh. Every week it’s a new application and every few months it’s some quantum leap.

Like others said, Midjourney and DallE are essentially photorealistic.

It seems to me that the next step is generative AI creating better and better assets.

And then of course you have video generation which is happening as well…

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#112

Post by kranke155 »

worldsayshi wrote: post #58
kranke155 wrote: post #7

My chosen profession (3D / filmmaking) feels like being in some kind of combat trench at the moment. Both fascinating and scary

I don't see that LLM's have come that much further in 3D animation than programming in this regard: It can spit out bits and pieces that looks okay in isolation but a human need to solve the puzzle. And often solving the puzzle means rewriting/redoing most of the pieces. We're safe for now but we should learn how to leverage the new tech.

The reason AI generative tools are faster to become useful in artistic areas is that in the arts you can take “errors” as style.

Doesn’t apply too much to mesh generation but was certainly the case in image gen. Mistakes that wouldn’t fly for a human artist (hands) were just accepted as part of AIgen.

So these areas are much less strict about precision than coding. Making these tools much more capable are replacing artists in some tasks than CoPilot is for coders atm.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#113

Post by eropple »

Art9681 wrote: post #102
LarsDu88 wrote: post #47

As a machine learning engineer who dabbles with Blender and hobby gamedev, this is pretty impressive, but not quite to the point of being useful in any practical manner (as far as the limited furniture examples are concerned. A competent modeler can make these types of meshes in under 5 minutes, and you still need to seed the generation with polys. I imagine the next step will be to have the seed generation controlle…

Just like a competent developer can use LLMs to bootstrap workflows, a competent model will soon have tools like this as part of their normal workflow. A casual user would be able to do things that they otherwise wouldnt have been able to. But an expert in the ML model's knowledge domain can really make it shine. I really believe that the more experienced you are in a particular use case, the more use you can get out…

I think eventually it may settle into what you describe. I don't think it's guaranteed, and I fear that there will be a pretty huge amount of damage done before that by the hype freaks whose real interest isn't in making artists more productive, but in rendering them (and other members of the actually-can-do-a-thing creative class) unemployed.

The pipeline problem also exists: if you need to still have the skillsets you build up through learning the craft, you still need to have avenues to learn the craft--and the people who already have will get old eventually.

There's a golden path towards a better future for everybody out of this, but a lot of swamps to drive into instead without careful forethought.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#115

Post by orbital-decay »

kranke155 wrote: post #111
orbital-decay wrote: post #74

Earlier quoted context omitted.

I don't know, 3D CGI has already been moving at the breakneck speed for the last three decades without any AI. Today's tools are qualitatively different (sculpting, simulation, auto-rigging etc etc etc).

3D CGI has gotten faster, but I haven’t seen any qualitative jump for quite some time. IMO the last time a major tech advance was visible was Davy Jones on the Pirates films. That was a fully photorealistic animated character that was plausible as a hero character in a major feature. That was a breakthrough. After that a lot of refinement and speeding up. This is different. I have some positivity about it, but it’s g…

Both DE3 and MJ are essentially toys for single random pictures, unusable in a professional setting. DALL-E in particular has really bad issues with quality, and while it follows the prompt well it also rewrites it so it's barely controllable. Midjourney is RLHF'd to death.

What you want for asset creation is not photorealism, but style and concept transfer, multimodal controllability (text alone is terrible at expressing artistic intent), and tooling. And tooling isn't something that is developed quickly (although there were several rapid breakthroughs in the past, for example ZBrush).

Most of the fancy demos you hear about sound good on paper, but don't really go anywhere. Academia is throwing shit at the wall to see what sticks, this is its purpose, especially when practice is running ahead of theory. It's similar to building airplanes before figuring out aerodynamics (which happened long ago): watching a heavier-than-air thing fly is amazing, until you realize it's not very practical in the current form, or might even kill its brave inventor who tried to fly it.

If you look at the field closely, most of the progress in visual generative tooling happens in the open source community; people are trying to figure out what works in real use and what doesn't. Little is being done in big houses, at least publicly and for now, as they're more interested in a DC-3 than a Caproni Ca.60. The change is really incremental and gradual, similarly to the current mature state of 3D. Paradigms are different but they are both highly technical and depend on academic progress. Once it matures, it's going to become another skill-demanding field.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#116

Post by legel »

shaileshm wrote: post #4

This is what a truly revolutionary idea looks like. There are so many details in the paper. Also, we know that transformers can scale. Pretty sure this idea will be used by a lot of companies to train the general 3D asset creation pipeline. This is just too great. "We first learn a vocabulary of latent quantized embeddings, using graph convolutions, which inform these embeddings of the local mesh geometry and topolog…

It's cool, it's also par for the field of 3D reconstruction today. I wouldn't describe this paper as particularly innovative or exceptional.

What do I think is really compelling in this field (given that it's my profession)?

This has me star-struck lately -- 3D meshing from a single image, a very large 3D reconstruction model trained on millions of all kinds of 3D models... https://yiconghong.me/LRM/

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#117

Post by willy_k »

reubenmorais wrote: post #96
taneq wrote: post #69

Earlier quoted context omitted.

Millions of people generating millions of images (some of them even useful!) using Dall-E and Stable Diffusion would say otherwise. A skilled digital artist could create most of these images in an hour or two, I’d guess… but ‘most people’ certainly could not, and it turns out that these people really want to.

Are those millions of people actually creating something of lasting value, or just playing around with a new toy?

A lot, but how many people will start with the latter but find themselves (capable of) doing the former?

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#118

Post by bradleyishungry »

valine wrote: post #26

Even if this is “only” mesh autocomplete, it is still massively useful for 3D artists. There’s a disconnect right now between how characters are sculpted and how characters are animated. You’d typically need a time consuming step to retopologize your model. Transformer based retopology that takes a rough mesh and gives you clean topology would be a big time saver. Another application: take the output of your gaussian…

sorry to tell you, but there’s no way anything will be generating clean topology for characters for a long long time.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#119

Post by willy_k »

johnnyanmac wrote: post #94
hipadev23 wrote: post #87

Earlier quoted context omitted.

> but it'll probably be 5+ years before this is commercially available I think you should look at the progress of image, text, and video generation over the past 12 months and re-asses your timeline prediction.

Availability =/= viability. I'm sure as we speak some large studios are already leveraging this work or are close to leveraging it. But this stuff trickles down to the public very slowly. Because indies aren't a good audience to sell what is likely an expensive tech that is focused on mid-large scale production.

Yes but no, none of that really describes current development.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#120

Post by johnnyanmac »

willy_k wrote: post #119
johnnyanmac wrote: post #94

Earlier quoted context omitted.

Availability =/= viability. I'm sure as we speak some large studios are already leveraging this work or are close to leveraging it. But this stuff trickles down to the public very slowly. Because indies aren't a good audience to sell what is likely an expensive tech that is focused on mid-large scale production.

Yes but no, none of that really describes current development.

perhaps, but I was responding to

>Most people are not a competent . Most people don't know a competent or can't afford to hire one.

emphasis mine. Affordability doesn't have much to do with capabilities, but it is a strong factor to consider for an indie dev. Devs in fields (games, VFX) that don't traditionally pay well to begin with.

Post reply on HN