Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeshGPT: Generating triangle meshes with decoder-only transformers

nihalsid.github.io

Post reply on HN
Print view View original

131–140 of 166 posts

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#131

Post by valine »

bradleyishungry wrote: post #118
valine wrote: post #26

Even if this is “only” mesh autocomplete, it is still massively useful for 3D artists. There’s a disconnect right now between how characters are sculpted and how characters are animated. You’d typically need a time consuming step to retopologize your model. Transformer based retopology that takes a rough mesh and gives you clean topology would be a big time saver. Another application: take the output of your gaussian…

sorry to tell you, but there’s no way anything will be generating clean topology for characters for a long long time.

There’s no shortage of 3D mesh data to train on. Who to say scaling up the parameter count won’t allow for increasingly intricate topology the same way scaling language models improved reading comprehension.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#132

Post by user432678 »

adventured wrote: post #108
worldsayshi wrote: post #58

Earlier quoted context omitted.

I don't see that LLM's have come that much further in 3D animation than programming in this regard: It can spit out bits and pieces that looks okay in isolation but a human need to solve the puzzle. And often solving the puzzle means rewriting/redoing most of the pieces. We're safe for now but we should learn how to leverage the new tech.

> We're safe for now Some are safe for several years (3-5), that's it. During that time it's going to wreck the bottom tiers of employees and progressively move up the ladder. GPT and the equivalent will be extraordinary at programming five years out. It will end up being a trivially easy task for AI in hindsight (15-20 years out), not a difficult task. Have you seen how far things like MidJourney, Dalle, Stable Diff…

Yeah, I’d better buy more wedding cakes in advance.

https://xkcd.com/605

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#134

Post by th0ma5 »

rawrawrawrr wrote: post #64
th0ma5 wrote: post #51

Earlier quoted context omitted.

This is a very underrated comment... As with any tech demo, I'd they don't show it, it can't do it. It is very very easy to imagine a generalization of these things to other purposes, which, if it could do it, would be a different presentation.

It's research, not meant for commercialization. The main point is in the process, not necessarily the output.

What? If the research doesn't show it, it can't do it, is my point, or else they would've put it in their research.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#136

Post by Mizza »

Great work. But I don't get from the demo how it knows what object to autocomplete the mesh with - if you give it four posts as an input, how does it know to autocomplete as a table and not a dog?

So maybe the next step is something like CLIP, but for meshes? CLuMP?

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#137

Post by hackerlight »

BrokrnAlgorithm wrote: post #130
ipsum2 wrote: post #126

Earlier quoted context omitted.

We're not stuck on meshes. Check out neural radiance fields as an alternative.

I was referring to being stuck with having to create simple / low tri polygonal meshes as opposed to using complex poly meshes such as photogrammetry would provide. The paper specifically addresses clean low poly meshes as opposed to what they call complex iso surfaces created by photogrammetry and other methods

Lots of polys is bad for performance. For a flat object like a table you want that to be low poly. Parallax can also help to give a 3D look without increasing poly count.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#138

Post by wolfgang805 »

beebeepka wrote: post #76

Games and pretty much any other experience being generated by AI is obvious to anyone paying attention at this point. But how would it work. Are current ai generated images and videos using rasterisation? Will they use rasterisation, path tracing or any other traditional rendering technique, or is will it be an entirely different thing.

Why would a video or an image, something generated without a mesh, be using rasterization?

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#139

Post by kranke155 »

orbital-decay wrote: post #115
kranke155 wrote: post #111

Earlier quoted context omitted.

3D CGI has gotten faster, but I haven’t seen any qualitative jump for quite some time. IMO the last time a major tech advance was visible was Davy Jones on the Pirates films. That was a fully photorealistic animated character that was plausible as a hero character in a major feature. That was a breakthrough. After that a lot of refinement and speeding up. This is different. I have some positivity about it, but it’s g…

Both DE3 and MJ are essentially toys for single random pictures, unusable in a professional setting. DALL-E in particular has really bad issues with quality, and while it follows the prompt well it also rewrites it so it's barely controllable. Midjourney is RLHF'd to death. What you want for asset creation is not photorealism, but style and concept transfer, multimodal controllability (text alone is terrible at expre…

With respect, I disagree with almost everything you said.

The idea that somehow “AI isn’t art directable” is one I keep hearing, but I remain unconvinced this is somehow an unsolvable problem.

The idea that AIgen is unusable at the moment for professional work doesn’t hold up to my experience since I now regularly use Photoshop’s gen feature.

Re: MeshGPT: Generating triangle meshes with decoder-only transformers

#140

Post by eurekin »

LarsDu88 wrote: post #47

As a machine learning engineer who dabbles with Blender and hobby gamedev, this is pretty impressive, but not quite to the point of being useful in any practical manner (as far as the limited furniture examples are concerned. A competent modeler can make these types of meshes in under 5 minutes, and you still need to seed the generation with polys. I imagine the next step will be to have the seed generation controlle…

I can imagine one usecase, in a typical architecture design, where the architect creates a design and always faces this stumbling block, when wanting to make it look as lively as possible: sprinkling a lot of convincing assets everywhere.

As they are generated, variations are much easier to come by easier, than buying a couple asset packs.

Post reply on HN