271–280 of 347 posts
Earlier quoted context omitted.
Given that the video is fully interactive and lets you move around (in a “world” if you will) I don’t think it’s a stretch to call it a world model. It must have at least some notion of physics, cause and effect, etc etc in order to achieve what it does.
No, it actually needs none of that.
Everyone here seems too caught up in the idea that Genie is the product, and that its purpose is to be a video game, movie, or VR environment. That is not the goal. The purpose of world models like Genie is to be the "imagination" of next-generation AI and robotics systems: a way for them to simulate the outcomes of potential actions in order to inform decisions.
I think you are anthropomorphising the AI too much. Imagination is inspired by reality, which AI does not have. Introducing a reality which the AI fully controls (looking beyond issues of vision and physics simulation) would only induce psychosis in the AI itself since false assumptions would only be amplified.
Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world. In thi…
This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter. This is one of the core tenets of Buddhism, and it's also expounded o…
Earlier quoted context omitted.
Why is it a dead end, you don’t meaningfully explain that. These models look like you can interact with them and they seem to replicate physics models.
They don't though, they're hallucinated videos. They're feeding models tons and tons of 2D videos and hoping they figure out physics from them, instead of just using a game engine and having the LLM write something up that works 100% of the time.
Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world. In thi…
Personally I often catch myself making reading mistakes and knowing for a fact that the mistake wasn't just conceptual, but an actual visual error where my brain renders the wrong word. Sometimes it's very obvious because the effect will last for seconds before my vision "snaps" back into reality and the word/phrase changes.
I first noticed this phenomenon in my subjective experience whenever I was 5 and started playing Pokémon. For many months, I thought Geodude was spelled and pronounced Gordude, until my neighbor said the name correctly one day and it "unlocked" my brain's ability to see the word spelled correctly.
The effect is so strong sometimes that I can close my eyes and imagine a few different moments in my life, even as a child, where my brain suddenly "saw" the right word while reading and it changed before my eyes.
Earlier quoted context omitted.
Let me introduce you to Idealism And more specifically Analytic Idealism https://youtu.be/P-rXm7Uk9Ys?si=q7Kefl7PbYfGiChZ Google DeepMind’s Project Genie is being framed as a “world model.” Given a text prompt, it generates a coherent, navigable, photorealistic world in real time. An agent can move through it, act within it, and the world responds consistently. Past interactions are remembered. Physics holds. Cause a…
It's pretty clear you used an LLM to write that, given your post history. I'm not sure that's allowed here, but at least put a disclaimer.
- but does it mean I don’t believe all the words written above are valid? No absolutely not.
I reviewed and copyedited what I posted and the meaning is exactly what I intended to post so I’m not sure what’s the issue here
If we use LLMs to expound on our own thoughts is it a crime? They are literal masters of wordplay and rote clarification on complex topics so I think this is a very legitimate use-case for them, since I was going for clarity as an objective- esp considering the topic
Also none of my previous posts were LLM written (including this one)
People are a little over-sensitive on this topic these days
Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world. In thi…
Let me introduce you to Idealism And more specifically Analytic Idealism https://youtu.be/P-rXm7Uk9Ys?si=q7Kefl7PbYfGiChZ Google DeepMind’s Project Genie is being framed as a “world model.” Given a text prompt, it generates a coherent, navigable, photorealistic world in real time. An agent can move through it, act within it, and the world responds consistently. Past interactions are remembered. Physics holds. Cause a…
For example, you can spin around, or change position, or close your eyes, and you're still you. As you navigate and interact with the evolving universe, the only continual, relatively unchanging part of the experience is what your brain uses to differentiate itself from the rest of your perceptions.
Earlier quoted context omitted.
Soft disagree; if you wanted imagination you don't need to make a video model. You probably don't need to decode the latents at all. That seems pretty far from information-theoretic optimality, the kind that you want in a good+fast AI model making decisions. The whole reason for LLMs inferencing human-processable text, and "world models" inferencing human-interactive video, is precisely so that humans can connect in…
Sufficiently informative latents can be decoded into video. When you simulate a stream of those latents, you can decode them into video. If you were trying to make an impressive demo for the public, you probably would decode them into video, even if the real applications don't require it. Converting the latents to pixel space also makes them compatible with existing image/video models and multimodal LLMs, which (with…