Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Marble: A Multimodal World Model

worldlabs.ai

Post reply on HN
Print view View original

31–40 of 87 posts

Re: Marble: A Multimodal World Model

#31

Post by saretup »

Marshferm wrote: post #29

If biology operated this way animals would never have evolved. This is bunk, it has nothing to do with intelligence and everything to do with hyping the oxymoronic/paradox branded as spatial intelligence. It's a flimsy way of using binary to burn energy. Senses do not represent, if we needed them to in order to survive, we'd be dead before we never saw that branch, or that step, etc. This is the same mistaken approac…

Counterpoint: Humans visualize stuff in their minds before trying new things or when learning new concepts. An AI system with LLM based language center and a world model to visualize during training and inference would help it overlap more of human intelligence. Also it looks cool.

Edit: After seeing your edited (longer) comment, I have no idea what you’re talking about.

Re: Marble: A Multimodal World Model

#32

Post by godelski »

dvrp wrote: post #19

Isn't this a Gaussian Splat model? I work in AI and, to this day, I don't know what they mean by “world” in “world model”.

  > I work in AI and, to this day, I don't know what they mean by “world” in “world model”.
I have a PhD in ML and a B.S. in physics. What people in ML call a "world model" seems incredibly strange to me. With my physics hat on, a "world model" is pretty clear. It is "a physics." Mind you, there is not one physics, there are competing models and we're just at a point of time that models have converged up to quantum and gravity.

But "a physics" can be a model that describes any world, not just the one we live in. For ML models, this should be based on the data they're processing. Ideally we'd want this to be similar to our own, but if it is modeling a "world" where pi=3, then that's still "a physics".

The key points here are that a physics is a counterfactual description of the environment. You have to have language to formalize relationships between objects. In standard physics (and most of science) we use math[0], though we use several languages (different algebras, different groups, different branches, etc) of math to describe different phenomena. But the point is that an equation is designed to be the maximum compression of that description. I don't really care if you use numbers or symbols, what matters is if you have counterfactual, testable, consistent, and concise descriptions of "the world".

Oddly enough, there are a lot of physicists and former physicists that work in ML but it is fairly uncommon for them to be working on "world modeling." I can tell you from my own experience talking to people who research world models that they respond to my concerns as "we just care if it works" as if that is also not my primary concern. Who the fuck isn't concerned with that? Philosophers?

[0] It can be easy to read "The Unreasonable Effectiveness of Mathematics in the Natural Sciences" out of context as we're >60 years past where math has been the lingua Franca of science. But math is a language, we invented it, and it should be surprising that this language is so powerful that we can work out the motion of the stars from a piece of paper. Math is really the closest thing we have to a magical language https://web.archive.org/web/20210212111540/http://www.dartmo...

Re: Marble: A Multimodal World Model

#33

Post by echelon »

Marshferm wrote: post #29

If biology operated this way animals would never have evolved. This is bunk, it has nothing to do with intelligence and everything to do with hyping the oxymoronic/paradox branded as spatial intelligence. It's a flimsy way of using binary to burn energy. Senses do not represent, if we needed them to in order to survive, we'd be dead before we never saw that branch, or that step, etc. This is the same mistaken approac…

I don't know who you are or why you're so negative, but I have an immediate use case for this and I'll come back and post a "Show HN" on this.

This is immensely useful tech for blocking out consistent scenes, eg. for video generation.

Beyond entertainment, this is going to be huge in the field of robotics.

I wish you could see what I see.

Re: Marble: A Multimodal World Model

#34

Post by dmarcos »

echelon wrote: post #33
Marshferm wrote: post #29

If biology operated this way animals would never have evolved. This is bunk, it has nothing to do with intelligence and everything to do with hyping the oxymoronic/paradox branded as spatial intelligence. It's a flimsy way of using binary to burn energy. Senses do not represent, if we needed them to in order to survive, we'd be dead before we never saw that branch, or that step, etc. This is the same mistaken approac…

I don't know who you are or why you're so negative, but I have an immediate use case for this and I'll come back and post a "Show HN" on this. This is immensely useful tech for blocking out consistent scenes, eg. for video generation. Beyond entertainment, this is going to be huge in the field of robotics. I wish you could see what I see.

I’m really curious to see what you are cooking. What field are you in?

Re: Marble: A Multimodal World Model

#35

Post by echelon »

padolsey wrote: post #22
dcl wrote: post #20

What happens when you prompt one of these kind of models with de_dust? Will it autocomplete the rest of the map? edit: Just tried it and it doesn't, but it does a good job of creating something like a CS map.

>What happens when you prompt one of these kind of models with de_dust? Presumably de_dust2

You both should check out DiamondWM. It runs on Ubuntu and I think Windows, presuming you have an Nvidia GPU. It's exactly what you're talking about.

I linked it elsewhere in this thread.

Re: Marble: A Multimodal World Model

#36

Post by Marshferm »

saretup wrote: post #31
Marshferm wrote: post #29

If biology operated this way animals would never have evolved. This is bunk, it has nothing to do with intelligence and everything to do with hyping the oxymoronic/paradox branded as spatial intelligence. It's a flimsy way of using binary to burn energy. Senses do not represent, if we needed them to in order to survive, we'd be dead before we never saw that branch, or that step, etc. This is the same mistaken approac…

Counterpoint: Humans visualize stuff in their minds before trying new things or when learning new concepts. An AI system with LLM based language center and a world model to visualize during training and inference would help it overlap more of human intelligence. Also it looks cool. Edit: After seeing your edited (longer) comment, I have no idea what you’re talking about.

It’s irrelevant - it has nothing analogous to mental imagery. It’s pseudoscience.

Edit - of course you have no idea, you have no grasp of the oscillatory-dynamic origins of consciousness, nor does it seem anyone in AI.

Re: Marble: A Multimodal World Model

#37

Post by Marshferm »

echelon wrote: post #33
Marshferm wrote: post #29

If biology operated this way animals would never have evolved. This is bunk, it has nothing to do with intelligence and everything to do with hyping the oxymoronic/paradox branded as spatial intelligence. It's a flimsy way of using binary to burn energy. Senses do not represent, if we needed them to in order to survive, we'd be dead before we never saw that branch, or that step, etc. This is the same mistaken approac…

I don't know who you are or why you're so negative, but I have an immediate use case for this and I'll come back and post a "Show HN" on this. This is immensely useful tech for blocking out consistent scenes, eg. for video generation. Beyond entertainment, this is going to be huge in the field of robotics. I wish you could see what I see.

This is arbitrary, it has nothing to do with specifics or coherence. Study it carefully particularly as it relates to general robitics.

Re: Marble: A Multimodal World Model

#38

Post by unconstrastive »

kkukshtel wrote: post #30

Incredibly disappointing release, especially for a company with so much talent and capital. Looking at the worlds generated here https://marble.worldlabs.ai/ it looks a lot more like they are just doing image generation for a multiview stereo 360 panoramas and then reprojecting that into space. The generations exhibit all the same image artifacts that come from this type of scanning/reconstruction work, all the same…

To be fair, multiview-consistent diffusion is extremely hard - it's an accomplishment of it's own right to get right, and still very useful. "World model" is probably a misnomer though (what even is a world model?). Their recent work on frame gen models is probably a bit closer to an actual world model in the traditional sense (https://www.worldlabs.ai/blog/rtfm).

Re: Marble: A Multimodal World Model

#39

Post by bee_rider »

lofties wrote: post #6
girfan wrote: post #5

This seems very interesting. Timely, given that Yann LeCun's vision also seems to align with world models being the next frontier: https://news.ycombinator.com/item?id=45897271

An established founder makes claims X is the new frontier. X receives hundreds of millions in funding. Other less established founders claim they are working on X too. VCs suffering from terminal FOMO pump billions more into X. X becomes the next frontier. The previous frontiers are promptly forgotten about.

What’s it going to do, take away funds from the otherwise extremely prudent AI sector?

Re: Marble: A Multimodal World Model

#40

Post by voodooEntity »

padolsey wrote: post #23
dvrp wrote: post #19

Isn't this a Gaussian Splat model? I work in AI and, to this day, I don't know what they mean by “world” in “world model”.

Yeh I still don't think there's a fixed definition of what a world model is or in what modality it will emerge. I'm unconvinced it will emerge as a satisfying 3d game-like first-person walkthrough.

Ye but there wont be, same as with "agi" and "ai" depends on whom you are asking *shrug
Post reply on HN