41–50 of 430 posts
So, diffusion models are game engines as long as you already built the game? You need the game to train the model. Chicken. Egg?
Like if I kill an enemy in some room and walk all the way across the map and come back, would the body still be there?
There is no text conditioning provided to the SD model because they removed it, but one can imagine a near future where text prompts are enough to create a fun new game! Yes they had to use RL to learn what DOOM looks like and how it works, but this doesn’t necessarily pose a chicken vs egg problem. In the same way that LLMs can write a novel story, despite only being trained on existing text. IMO one of the biggest…
I would expect something in this realm to be a little better at not being visually inconsistent when you look away and look back. A red monster turning into a blue friendly etc.
Earlier quoted context omitted.
>Technically speaking, isn't this the greatest possible anti-Doom When I read this part I thought you were going to say because you're technically not running Doom at all. That is, instead of running Doom without Doom's original hardware/software environment (by porting it), you're running Doom without Doom itself.
Pierre Menard, Author of Doom.
It's insane that that this works, and that it works fast enough to render at 20 fps. It seems like they almost made a cross between a diffusion model and an RNN, since they had to encode the previous frames and actions and feed it into the model at each step. Abstractly, it's like the model is dreaming of a game that it played a lot of, and real time inputs just change the state of the dream. It makes me wonder if hu…
Image is 2D. Video is 3D. The mathematical extension is obvious. In this case, low resolution 2D (pixels), and the third dimension is just frame rate (discrete steps). So rather simple.
How does the model “remember” the whole state of the world? Like if I kill an enemy in some room and walk all the way across the map and come back, would the body still be there?
So, diffusion models are game engines as long as you already built the game? You need the game to train the model. Chicken. Egg?
Earlier quoted context omitted.
From the article: We present GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with a complex environment over long trajectories at high quality . The demo is actual gameplay at ~20 FPS.
It confused me that their stated evaluations by humans are comparing video clips rather than evaluating game play.
How does the model “remember” the whole state of the world? Like if I kill an enemy in some room and walk all the way across the map and come back, would the body still be there?
Watch closely in the videos and you'll see that enemies often respawn when offscreen and sometimes when onscreen. Destroyed barrels come back, ammo count and health fluctuates weirdly, etc. It's still impressive, but its not perfect in that regard.