The demo button, and most of the features mentioned, dont seem to work for me, on edge or chrome. Project sounds really interesting, so I wish I could try!
MIRA: Multiplayer Interactive World Models Trained on Rocket League
31–35 of 35 posts
Re: MIRA: Multiplayer Interactive World Models Trained on Rocket League
#32Hey all, happy to see this here! This was a colab between General Intuition (that I’m part of), Kyutai and Epic Games. You can read plenty of details in the blog post and tech report but the TLDR is that we trained a multiplayer world model on 10k hours of Rocket League data. We optimized it to be playable at 20fps on a single GPU. So what you see in the demo is fully generated: there’s no graphics or physics engine.…
Very cool, and publishing a slice of the dataset and all of the training code is fantastic, but if reproducing the model and the video representation codec is encouraged, why not open source the models or at least some variant of them?
I'd be interested in seeing if fine-tunes that include human gameplay data would be possible.
Re: MIRA: Multiplayer Interactive World Models Trained on Rocket League
#33It feels like playing on a very slow computer. Except that sometimes it just randomly decides you pressed the flip button. Really impressive.
I played again but this time I drove my car backwards and the model does much, much worse in this weird situation. It often doesn't do what I input at all and keeps straightening itself out. And on kick-off you always go for it, even if you don't.
Re: MIRA: Multiplayer Interactive World Models Trained on Rocket League
#34It feels like playing on a very slow computer. Except that sometimes it just randomly decides you pressed the flip button. Really impressive.
I played again but this time I drove my car backwards and the model does much, much worse in this weird situation. It often doesn't do what I input at all and keeps straightening itself out. And on kick-off you always go for it, even if you don't.
Playing backward is actually a great idea for testing generalization
Re: MIRA: Multiplayer Interactive World Models Trained on Rocket League
#35Václav here from the team, we're happy to answer questions :) The most surprising part to me is the auto-recovery behavior we mention at the end of the blog post, since any other model I've seen always stays diverged once it goes off the rails once. But MIRA really doesn't like to be out-of-distribution. To be completely honest we're not entirely sure why this happens.
I'm insanely impressed with MIRA! I mean WOW, I've been doing a lot of diffusion world model experiments and I had thought of trying stuff like "visual anchors" to support some form of long form continuity but you guys really blew the field out of the water. My intuition(i've still yet to fully read the research paper i apologize) is that feeding in 4 highly dynamic(yet correlated, due to focusing on the ball) perspectives for each frame is what gives the incredible coherence. As said in the blog and paper, essentially infinite rollout. Thank you, and all of the team, for making your work public! I'm not above begging for an internship just to be clear https://github.com/blackfirebitcoin/hyrule-dreamer-wm