311–320 of 430 posts
Earlier quoted context omitted.
It is just video. There are no external interactions. Heck, it is far simpler than video, because the point of view and frame is fixed.
? I highly suggest you to read the paper briefly before commenting on the topic. The whole point is that it's not just generating a video.
Earlier quoted context omitted.
I think you're mistaken. The abstract says it's interactive, "We present GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction" Further - "a diffusion model is trained to produce the next frame, conditioned on the sequence of past frames and actions." specifically "and actions" User input is being fed into this system and subsequent frames take that into account. The us…
No, I am not. The interaction is part of the training, and is used during inference, but it is not including during the process of generation.
It's trained on a large set of data in which agents played DOOM and video samples are given to users for evaluation, but users are not feeding inputs into the simulation in real-time in such a way as to be "playing DOOM" at ~20FPS.
There are some key phrases within the paper that hint at this such as "Key questions remain, such as ... how games would be effectively created in the first place, including how to best leverage human inputs" and "Our end goal is to have human players interact with our simulation.", but mostly it's just the omission of a section describing real-time user gameplay.
Earlier quoted context omitted.
Are you saying if I turn around, I’ll be surprised at what I find ? I don’t feel like this is accurate at all.
Not exactly, but our representation of what's behind us is a lot more sparse than we would assume. That is, I might not be surprised by what I see when I turn around, but it could have changed pretty radically since I last looked, and I might not notice. In fact, an observer might be quite surprised that I missed the change. Objectively, Simons and Chabris (and many others) have a lot of data to support these ideas.…
Even having a clue why I'm linking this, I virtually guarantee you won't catch everything.
And even if you do catch everything... the real thing to notice is that you had to look. Your brain does not flag these things naturally. Dreams are notorious for this sort of thing, but even in the waking world your model of the world is much less rich than you think. Magic tricks like to hide in this space, for instance.
What most programmers don't understand, that in the very near future, the entire application will be delivered by an AI model, no source, no text, just connect to the app over RDP. The whole app will be created by example, the app developer will train the app like a dog trainer trains a dog.
So... https://websim.ai except over pixels instead of in your browser?
After some discussion in this thread, I found it worth pointing out that this paper is NOT describing a system which receives real-time user input and adjusts its output accordingly, but, to me, the way the abstract is worded heavily implied this was occurring. It's trained on a large set of data in which agents played DOOM and video samples are given to users for evaluation, but users are not feeding inputs into the…
Earlier quoted context omitted.
But does it need to be frame-based? What if you combine this with an engine in parallel that provides all geometry including characters and objects with their respective behavior, recording changes made through interactions the other model generates, talking back to it? A dialogue between two parties with different functionality so to speak. (Non technical person here - just fantasizing)
In that scheme what is the NN providing that a classical renderer would not? DOOM ran great on an Intel 486, which is not a lot of computer.
It always blew my mind how well it worked on a 33 Mhz 486. I'm fairly sure it ran at 30 fps in 320x200. That gives it just over 17 clock cycles per pixel, and that doesn't even include time for game logic.
My memory could be wrong, though, but even if it required a 66 Mhz to reach 30 fps, that's still only 34 clocks per pixel on an architecture that required multiple clocks for a simple integer add instruction.
After some discussion in this thread, I found it worth pointing out that this paper is NOT describing a system which receives real-time user input and adjusts its output accordingly, but, to me, the way the abstract is worded heavily implied this was occurring. It's trained on a large set of data in which agents played DOOM and video samples are given to users for evaluation, but users are not feeding inputs into the…
> Figure 1: a human player is playing DOOM on GameNGen at 20 FPS.
The abstract is ambiguously worded which has caused a lot of confusion here, but the paper is unmistakably clear about this point.
Kind of disappointing to see this misinformation upvoted so highly on a forum full of tech experts.
Earlier quoted context omitted.
If a generic human glances at an unfamiliar screen/wall/room, can they accurately, pixel-perfectly reconstruct every single element of it? Can they do it for every single screen they have seen in their entire lives?
I never said pixel perfect, but I would be surprised if whole objects , like flaming lanterns suddenly appeared. What this demo demonstrates to me is how incredible willing we are to accept what seems familiar to us as accurate. I bet if you look closely and objectively you will see even more anomalies. But at first watch, I didn’t see most errors because I think accepting something is more efficient for the brain.