Live data from Hacker News

Fable 5.1 World Modeling

github.com

71–80 of 115 posts

Re: Fable 5.1 World Modeling

#71
post #37

Earlier quoted context omitted.

> Opus 5 is just as good for this tbh and cheaper. It depends on the fidelity, there's a threshold slightly over "low poly" that Opus cannot get over. Once you get into creating foliage l-systems, or physical mob animation, or a house with realistic appliances, Opus is under the convergence threshold for a world model no matter how much time you give it, it will flail around and say it's done even though it's nowhere…

> then fit it back to the world with PCA + BB's Can you expand on this? I've tried several AI 3d generators that claim low-poly output, and they're all far off. None of them decide what belongs in the mesh vs. what should be baked in with the texture, so the poly budget lands in the wrong places.

What I'm suggesting is if you want a certain semantic part of the scene to have a specific design or fidelity (say an interactive object, point of interest, character, or parts thereof), you can generate it via image and then use principal component analysis, bounding boxes, and a physics engine like Jolt to place and orient it programmatically into the right "socket" in your world so it aligns and joins correctly. The LLM can generate the high level spatial/semantic structure, and you can plug the assets into the right space. That works both at scene level and more locally (say a character's equipment they are wearing or holding).

This kind of decomposition helps to get the art composed the way you want it, though it won't help with a runtime poly budget.

But if you are intentionally doing low-poly style, it is doubtful you are (at least technologically) limited by having too many vertices on assets; you're limited by draws and submits and rendering architecture. You can push millions of animated polys with bells and whistles, postprocessing, physics etc, in the browser, at 90 FPS, on a macbook. You can have thousands of objects, but you have to cull/instance/share materials/atlas/etc, you can't just do `new THREE.Mesh` for each object -- which is what the LLMs will naively do by default unless you ask otherwise.

If resource size is a problem, meshes compress/quantize well with things like meshopt/DRACO.

Re: Fable 5.1 World Modeling

#74

Fundamentally it’s an image model predicting out frames of a sequence. It’s not enough to call it just an image model though. Not quite video either because it’s so specific to rendering images of the world. Thus, “world model”? The problem I have with “world” is it should imply so much more depth. To me this is actually a “POV image model” or “First-person perspective model” (maybe we call this an FPM). There could…

I think perhaps you've misunderstood what the linked repo actually is? It seems to indeed be a 3d model of an area. It's not an image model, and nothing to do with frames.

Re: Fable 5.1 World Modeling

#75
what's going on with these overlays? the readme is presenting those as though they are evidence of the accuracy of the project, but its completely unable to line up the shots

https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...

https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...

Re: Fable 5.1 World Modeling

#77

I have experimented with this type of modeling for a RTS game I'm building: https://playwarpact.com Few things: 1. Opus 5 is just as good for this tbh and cheaper. 2. They don't generate optimized 3d models. They have high poly count for simple geometries. 3. A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes of the 3d model and then bake textures that include a l…

Uh, I tried feeding 5.6 Sol High my 2D DXF drawings of my house floor layout & house exterior to generate 3D visualization within blender via MCP... and it was miserable. No understanding on how windows attach to walls, misaligned bearing walls between floors, gaps all around the places, chimney not continuous, windows don't line up and are in wrong places, house parts swapped. And still struggling when I explain those things - that it is a house and real-life architectural constraints should be taken into account. But I just had to keep prompting to fix this, that, redo this and that and then it breaks other things until I fed up.

Part of it is probably struggle to understand the DXF and mixing up "wall lines" with "dimension/helper lines" (I tried also feeding PDFs)

Anyways, if someone has an idea how to improve 3D modelling with AI or how to make understand 2D drawings to understand rooms/walls/square meters, etc - would be nice.

Re: Fable 5.1 World Modeling

#78

I have experimented with this type of modeling for a RTS game I'm building: https://playwarpact.com Few things: 1. Opus 5 is just as good for this tbh and cheaper. 2. They don't generate optimized 3d models. They have high poly count for simple geometries. 3. A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes of the 3d model and then bake textures that include a l…

Please do post the tutorial. Also, your RTS looks great, I will definitely check it out!

Re: Fable 5.1 World Modeling

#79
post #73

This is really cool, but $33 for a single generated world makes it hard to see this being useful for games just yet. Not to mention how this would go in a much larger project

$33 is expensive? Compared to what?

There are free open source tools that convert osm data into 3D models, for example https://osm2world.org/. Also you could use freely available national 3d point cloud data, at least for some areas. both are $0 options. btw I am not saying $33 dollars here is expensive; just giving you some comparisons

Re: Fable 5.1 World Modeling

#80
post #19

This reminds me of the experience VRML wanted to be 3 decades ago. The only difference is that links went to other worlds living on another server. But that seems trivial to implement with this.

VRML was incredible. It's a "hypertext" markup language but instead of documents it produces 3D environments. It's a shame it never really caught on.

There's some work going on at the Metaverse Standards Forum to progress these ideas with Web of Worlds (https://webofworlds.github.io/). It's scope is a lot broader than VRML, it covers assets, identity, avatars, payments, etc.

There's a link to the presentation at SIGGRAPH - https://youtu.be/N2_lb77gKQ8.

Post reply on HN