61–70 of 430 posts
How does the model “remember” the whole state of the world? Like if I kill an enemy in some room and walk all the way across the map and come back, would the body still be there?
Earlier quoted context omitted.
This is not "just" video, however. It's interactive in real time. Sure, you can say that playing is simply video with some extra parameters thrown in to encode player input, but still.
It is just video. There are no external interactions. Heck, it is far simpler than video, because the point of view and frame is fixed.
I highly suggest you to read the paper briefly before commenting on the topic. The whole point is that it's not just generating a video.
I'm wondering when people will apply this to other areas like the real world. Would it learn the game engine of the universe (ie physics)?
Earlier quoted context omitted.
This is not "just" video, however. It's interactive in real time. Sure, you can say that playing is simply video with some extra parameters thrown in to encode player input, but still.
It is just video. There are no external interactions. Heck, it is far simpler than video, because the point of view and frame is fixed.
Further - "a diffusion model is trained to produce the next frame, conditioned on the sequence of past frames and actions." specifically "and actions"
User input is being fed into this system and subsequent frames take that into account. The user is "actually" firing a gun.
The quest to run doom on everything continues. Technically speaking, isn't this the greatest possible anti-Doom, the Doom with the highest possible hardware requirement? I just find it funny that on a linear scale of hardware specification, Doom now finds itself on both ends.
> Technically speaking, isn't this the greatest possible anti-Doom, the Doom with the highest possible hardware requirement? Not really? The greatest anti-Doom would be an infinite nest of these types of models predicting models predicting Doom at the very end of the chain. The next step of anti-Doom would be a model generating the model, generating the Doom output.
Ah finally we are starting to see something gaming related. I'm curious as to why we haven't seen more of neural networks applied to games even in a completely experimental fashion; we used to have a lot of little experimental indie games such as Façade (2005) and I'm surprised we don't have something similar years after the advent of LLMs. We could have mods for old games that generate voices for the characters for…
You mean in real time? Or just in general?
There are a lot of mods that use AI-generated voices. I'll say it's the norm of modding community now.
It's insane that that this works, and that it works fast enough to render at 20 fps. It seems like they almost made a cross between a diffusion model and an RNN, since they had to encode the previous frames and actions and feed it into the model at each step. Abstractly, it's like the model is dreaming of a game that it played a lot of, and real time inputs just change the state of the dream. It makes me wonder if hu…
So, diffusion models are game engines as long as you already built the game? You need the game to train the model. Chicken. Egg?
here are some ideas: - you could build a non-real-time version of the game engine and use the neural net as a real-time approximation - you could edit videos shot in real life to have huds or whatever and train the neural net to simulate reality rather than doom. (this paper used 900 million frames which i think is about a year of video if it's 30fps, but maybe algorithmic improvements can cut the training requiremen…
Given a sufficient enough separation between these two, couldn't you basically boil the game/input logic down to an abstract game template? Meaning, you could just output a hash that corresponds to a specific combination of inputs, and then treat the resulting mapping as a representation of a specific game's inner workings.
To make it less abstract, you could save some small enough snapshot of the game engine's state for all given input sequences. This could make it much less dependent to what's recorded off of the agents' screens. And you could map the objects that appear in the saved states to graphics, in a separate step.
I imagine this whole system would work especially well for games that only update when player input is given: Games like Myst, Sokoban, etc.
What most programmers don't understand, that in the very near future, the entire application will be delivered by an AI model, no source, no text, just connect to the app over RDP. The whole app will be created by example, the app developer will train the app like a dog trainer trains a dog.