The "How we're building responsibly" section has nothing to do with acting responsibly. It should be called "Limitations" instead. Section reads LLM generated honestly.
Project Genie: Experimenting with infinite, interactive worlds
191–200 of 347 posts
Re: Project Genie: Experimenting with infinite, interactive worlds
#192Earlier quoted context omitted.
After a lifetime career in tech, I want to turn it all off.
Seriously though. These cute distractions have turned into world and culture eating monsters.
Can't wait to see this being abused to show that "the prisoners were treated humanely" or that "there are no protests in the capital." At what point to we hold engineers responsible for creating these machines? How many must suffer because no one was willing to say no?
Re: Project Genie: Experimenting with infinite, interactive worlds
#193Really great to see this released! Some interesting videos from early-access users: - https://youtu.be/15KtGNgpVnE?si=rgQ0PSRniRGcvN31&t=197 walking through various cities - https://x.com/fofrAI/status/2016936855607136506 helicopter / flight sim - https://x.com/venturetwins/status/2016919922727850333 space station, https://x.com/venturetwins/status/2016920340602278368 Dunkin' Donuts - https://youtu.be/lALGud1Ynhc?si=…
I was lucky enough to be an early tester, here's a brief video walking through the process of creating worlds, showing examples--walking on the moon, with Nasa photo as part of the prompt, being in 221B Baker street with Holmes and Watson, wandering through a night market in Taipei as a giant boba milk tea (note how the stalls are different, and sell different foods), and also exploring the setting of my award-nominated tabletop RPG.
https://www.youtube.com/watch?v=FyTHcmWPuJE
It's an experimental research prototype, but it also feels like a hint of the future. Feel free to ask any questions.
Re: Project Genie: Experimenting with infinite, interactive worlds
#194The more of this I see the more I want to spend time away from screens and doing those things I love to do in the real world.
Most people don't have access to anything particularly nice in the real world. It's reality privilege. Most of humanity will yearn for the worlds that AI will cook up for them, customized to their whims.
What a nonsensical take, good lord.
Re: Project Genie: Experimenting with infinite, interactive worlds
#195Really great to see this released! Some interesting videos from early-access users: - https://youtu.be/15KtGNgpVnE?si=rgQ0PSRniRGcvN31&t=197 walking through various cities - https://x.com/fofrAI/status/2016936855607136506 helicopter / flight sim - https://x.com/venturetwins/status/2016919922727850333 space station, https://x.com/venturetwins/status/2016920340602278368 Dunkin' Donuts - https://youtu.be/lALGud1Ynhc?si=…
Any thoughts about Project Genie?
On a technical level, this looks like the same diffusion transformer world model design that was shown in the Genie 3 post (text/memory/d-pad input, video output, 60sec max context, 720p, sub-10FPS control latency due to 4-frame temporal compression). I expect the public release uses a cheaper step-distilled / quantized version. The limitations seen in Genie 3 (high control latency, gradual loss of detail and drift towards videogamey behavior, 60s max rollout length) are still present. The editing/sharing tools, latency, cost, etc. can probably improve over time with this same model checkpoint, but new features like audio input/output, higher resolution, precise controls, etc. likely won't happen until the next major version.
From a product perspective, I still don't have a good sense of what the market for WMs will look like. There's a tension between serious commercial applications (robotics, VFX, gamedev, etc. where you want way, way higher fidelity and very precise controllability), vs current short-form-demos-for-consumer-entertainment application (where you want the inference to be cheap-enough-to-be-ad-supported and simple/intuitive to use). Framing Genie as a "prototype" inside their most expensive AI plan makes a lot of sense while GDM figures out how to target the product commercially.
On a personal level, since I'm also working on world models (albeit very small local ones https://news.ycombinator.com/item?id=43798757), my main thought is "oh boy, lots of work to do". If everyone starts expecting Genie 3 quality, local WMs need to become a lot better :)
Re: Project Genie: Experimenting with infinite, interactive worlds
#196Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world. In thi…
Another analogy that kinda fits in with what you're saying is the post-processing on smartphone "photos."
At what point does the processing become so strong that it's less a photograph and more a work of computational impressionism?
Re: Project Genie: Experimenting with infinite, interactive worlds
#197This could be the future of film. Instead of prompting where you don't know what the model will produce, you could use fine-grained motion controls to get the shot you are looking for. If you want to adjust the shot after, you could just checkpoint the model there, by taking a screenshot, and rerun. Crazy.
[deleted]
Re: Project Genie: Experimenting with infinite, interactive worlds
#198Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world. In thi…
I think this is pretty well established as far as neurologists are concerned and explains a lot of things. Like dreaming for instance.. just something like the model running without sensory input constraining it.
Re: Project Genie: Experimenting with infinite, interactive worlds
#199Earlier quoted context omitted.
This is a video model, not a world model. Start learning on this, and cascading errors will inevitably creep into all downstream products. You cannot invent data.
Given that the video is fully interactive and lets you move around (in a “world” if you will) I don’t think it’s a stretch to call it a world model. It must have at least some notion of physics, cause and effect, etc etc in order to achieve what it does.
[deleted]
Re: Project Genie: Experimenting with infinite, interactive worlds
#200Now I can't stop thinking about _The Experience Machine_ by Andy Clark. It theorizes that this is how humans navigate and experience the real world: Our brains generate what we think the world around is like and our senses don't so much directly process visual information but instead act like a kind of loss function for our internal simulations. Then we use that error to update our internal model of the world. In thi…
Also check out The Case Against Reality by Donald Hoffman