Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Marble: A Multimodal World Model

worldlabs.ai

Post reply on HN
Print view View original

41–50 of 87 posts

Re: Marble: A Multimodal World Model

#41

Post by saretup »

Marshferm wrote: post #36
saretup wrote: post #31

Earlier quoted context omitted.

Counterpoint: Humans visualize stuff in their minds before trying new things or when learning new concepts. An AI system with LLM based language center and a world model to visualize during training and inference would help it overlap more of human intelligence. Also it looks cool. Edit: After seeing your edited (longer) comment, I have no idea what you’re talking about.

It’s irrelevant - it has nothing analogous to mental imagery. It’s pseudoscience. Edit - of course you have no idea, you have no grasp of the oscillatory-dynamic origins of consciousness, nor does it seem anyone in AI.

histrionic and meretricious

Re: Marble: A Multimodal World Model

#42

Post by ehnto »

jtfrench wrote: post #27

I like that they distinguish between the collider mesh (lower poly) and the detailed mesh (higher poly). As a game developer I'm looking for: • Export low-poly triangle mesh (ideally OBJ or FBX format — something fairly generic, nothing too fancy) • Export texture map • Export normals • Bonus: export the scene as "de-structured" objects (e.g. instead of a giant world mesh with everything baked into it, separate expor…

Aren't the gausian splats the output here? Or are these worlds fully meshed and textured assets?

From my understanding, admittedly quite a shallow look so far, the model generates gaussian splats then from that could implement the collider.

I guess from the splat and the colliders you could generate actual assets that could be interactable/animated/have physics etc. Unsure, exciting space though! I just don't know how I would properly use this in a game, the examples are all quite on-rails and seem to avoid interacting too much with stuff in the environment.

Re: Marble: A Multimodal World Model

#43

Post by ehnto »

whizzter wrote: post #15
abixb wrote: post #8

As someone with barebones understanding of "world models," how does this differ from sophisticated game engines that generate three-dimensional worlds? Is it simply the adaptation of transformer architecture in generating the 3-D world v/s using a static/predictable script as in game engines (learned dynamics vs deterministic simulation mimicking 'generation')? Would love an explanation from SMEs.

Games are still mostly polygon based due to tooling (Even Unreal Nanite is a special variation of handling polygons), some engines have tried voxels (Teardown, Minecraft genererates polygons and would fall in the previous category as far as rendering goes) or even implict surface modes by composing SDF'y primitives (Dreams on Playstation and more recently unbound.io). All of these have fairly "exact" representations,…

Your later point is what makes me think this doesn't have comprehensive legs, just niche usage.

A typical game has thousands of hand placed nodes in 3D space, that do things like place lights, trigger story beats, account for physics and collisions etc. That wouldn't change with Gaussian splats, but if you needed to edit the world then even with deterministic generation, the whole world might change, and all your gameplay nodes are now misplaced.

That doesn't matter for some games, but I think it does matter for most.

Re: Marble: A Multimodal World Model

#44

Post by AlexisArgyriou »

whizzter wrote: post #15
abixb wrote: post #8

As someone with barebones understanding of "world models," how does this differ from sophisticated game engines that generate three-dimensional worlds? Is it simply the adaptation of transformer architecture in generating the 3-D world v/s using a static/predictable script as in game engines (learned dynamics vs deterministic simulation mimicking 'generation')? Would love an explanation from SMEs.

Games are still mostly polygon based due to tooling (Even Unreal Nanite is a special variation of handling polygons), some engines have tried voxels (Teardown, Minecraft genererates polygons and would fall in the previous category as far as rendering goes) or even implict surface modes by composing SDF'y primitives (Dreams on Playstation and more recently unbound.io). All of these have fairly "exact" representations,…

You could in theory combine point clouds and Nanite: cull sub-pixel points and generate geometry on the fly by filling the voids between remaining points with polygons. The main issue is bandwidth, GPUs are barely able to handle Nanite; and this would be at least an order of magnitude more complex to do at runtime. Nanite is doing a lot of offline precomputation, storing some sort of intermediate models etc.

Re: Marble: A Multimodal World Model

#45

Post by dandersch »

jtfrench wrote: post #28

Is Marble's definition of a "world model" the same as Yann LeCun's definition of a world model? And is that the same as Genie's definition of a world model?

Pretty sure it's used as a marketing term here. They train on images that you generate/give it, but the output of that training is not a model, it's a static 3d scene made up out of gaussian splats. You are not running inference on a model when traversing one of those scenes, you are just rendering the splats.

Re: Marble: A Multimodal World Model

#46

Post by anilgulecha »

This is going to take movie making to another level because now we can: 1. Generate a full scene 2. Generate a character with specific movements.

Combine these 2, and we can have moving cameras as needed (long takes). This is going to make storytelling very expressive.

Incredible times! Here's a bet: We'll have a AI superstar (A-list level) in the next 12 months.

Re: Marble: A Multimodal World Model

#47

Post by moi2388 »

anilgulecha wrote: post #46

This is going to take movie making to another level because now we can: 1. Generate a full scene 2. Generate a character with specific movements. Combine these 2, and we can have moving cameras as needed (long takes). This is going to make storytelling very expressive. Incredible times! Here's a bet: We'll have a AI superstar (A-list level) in the next 12 months.

An a-list level actor superstar within 12 months?

I’m willing to take that bet. Name any amount you’re willing to lose.

Before you agree: movies take more than 1 year to make and get published, and it takes more than 1 movie to make somebody an a-lister

Re: Marble: A Multimodal World Model

#48

Post by moi2388 »

keyle wrote: post #4

I'm floored. Incredible work. also check out their interactive examples on the webapp. It's a bit more rough around the edges but shows real user input/output. Arguably such examples could be pushed further to better quality output. e.g. https://marble.worldlabs.ai/world/b75af78a-b040-4415-9f42-6d... e.g. https://marble.worldlabs.ai/world/cbd8d6fb-4511-4d2c-a941-f4...

Very cool tech, extremely annoying voice over.

Re: Marble: A Multimodal World Model

#49

Post by qwertytyyuu »

anilgulecha wrote: post #46

This is going to take movie making to another level because now we can: 1. Generate a full scene 2. Generate a character with specific movements. Combine these 2, and we can have moving cameras as needed (long takes). This is going to make storytelling very expressive. Incredible times! Here's a bet: We'll have a AI superstar (A-list level) in the next 12 months.

the last few % that make a a-list actor an a-list actor is the hardest part, i would bet you that its going to take longer than 12 months
Post reply on HN