Please forgive my naivety, but are world models (once they are in a consumer-ready form) expected to outperform any currently existing LLM on these sorts of tasks (i.e. of the physical world)?
From what I have seen "world models" are more like video models. The idea is: from a video clip, predict the next frame, like LLMs predict the next word/token from a bit of text. The idea is that these models should be able to internalize the laws of physics and properties of objects just like LLMs do with grammar rules. To make training more efficient, frames from game engines with hardcoded physics are used. The ap…
The whole field of world models also have lighter approaches based on using latent representations as the target (and not pixel frames). LeWorldModel[0] and DINO-WM[1] are a good way to learn how these works.
Strangely, most of the startups in the space of world models are focused on video-generative world models.
A good application of world models is in robotics where an action policy generate a distribution of actions, you can then predict the future state of the world and use some reward policies to choose the "best" future state.