Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Diffusion models are real-time game engines

gamengen.github.io

Post reply on HN
Print view View original

291–300 of 430 posts

Re: Diffusion models are real-time game engines

#291

Post by dewarrn1 »

bamboozled wrote: post #270
dewarrn1 wrote: post #221

Earlier quoted context omitted.

Great observation. And not entirely unlike normal human visual perception which is notoriously vulnerable to missing highly salient information; I'm reminded of the "gorillas in our midst" work by Dan Simons and Christopher Chabris [0]. [0]: https://en.wikipedia.org/wiki/Inattentional_blindness#Invisi...

Are you saying if I turn around, I’ll be surprised at what I find ? I don’t feel like this is accurate at all.

Not exactly, but our representation of what's behind us is a lot more sparse than we would assume. That is, I might not be surprised by what I see when I turn around, but it could have changed pretty radically since I last looked, and I might not notice. In fact, an observer might be quite surprised that I missed the change.

Objectively, Simons and Chabris (and many others) have a lot of data to support these ideas. Subjectively, I can say that these types of tasks (inattentional blindness, change blindness, etc.) are humbling.

Re: Diffusion models are real-time game engines

#292

Post by Workaccount2 »

aithrowaway1987 wrote: post #81
w_for_wumbo wrote: post #13

Earlier quoted context omitted.

That feels like the endgame of video game generation. You select an art style, a video and the type of game you'd like to play. The game is then generated in real-time responding to each action with respect to the existing rule engine. I imagine a game like that could get so convincing in its details and immersiveness that one could forget they're playing a game.

Have you ever played a video game? This is unbelievably depressing. This is a future where games like Slay the Spire, with a unique art style and innovative gameplay simply are not being made. Not to mention this childish nonsense about "forget they're playing a game," as if every game needs to be lifelike VR and there's no room for stylization or imagination. I am worried for the future that people think they want t…

The problem is quite the opposite, that AI will be able to generate games so many game with so many play styles that it will totally dilute the value of all games.

Compare it to music gen algo's that can now produce music that is 100% indiscernible from generic crappy music. Which is insane given that 5 years ago it could maybe create the sound of something that maybe someone would describe as "sort of guitar-like". At this rate of progress it's probably not going to be long before AI is making better music than humans. And it's infinitely available too.

Re: Diffusion models are real-time game engines

#293

Post by aithrowaway1987 »

harha_ wrote: post #282

This is so sick I don't know what to say. I never expected this, aren't the implications of this huge?

I am struggling to understand a single implication of this! How does this generalize to anything other than other than playing retro games in the most expensive way possible? The very intention of this project is overfitting to data in a non-generalizable way! Maybe it's just pure engineering, that good ANNs are getting cheap and fast. But this project still seems to have the fundamental weaknesses of all AI projects:

- needs a huge amount of data, which a priori precludes a lot of interesting use cases

- flashy-but-misleading demos which hide the actual weaknesses of the AI software (note that the player is moving very haltingly compared to a real game of DOOM, where you almost never stop moving)

- AI nailing something really complicated for humans (98% effective raycasting, 98% effective Python codegen) while failing to grasp abstract concepts rigorously understood by fish (object permanence, quantity)

I am genuinely struggling to see this as a meaningful step forward. It seems more like a World's Fair exhibit - a fun and impressive diversion, but probably not a vision of the future. Putting it another way: unlike AlphaGo, Deep Blue wasn't really a technological milestone so much as a sociological milestone reflecting the apex of a certain approach to AI. I think this DOOM project is in a similar vein.

Re: Diffusion models are real-time game engines

#294

Post by calebh »

dtagames wrote: post #224

A diffusion model cannot be a game engine because a game engine can be used to create new games and modify the rules of existing games in real time -- even rules which are not visible on-screen. These tools are fascinating but, as with all AI hype, they need a disclaimer: The tool didn't create the game. It simply generated frames and the appearance of play mechanics from a game it sampled (which humans created).

One thing I'd like to see is to take a game rendered with low poly assets (or segmented in some way) and use a diffusion model to add realistic or stylized art details. This would fix the consistency problem while still providing tangible benefits.

Re: Diffusion models are real-time game engines

#295

Post by Workaccount2 »

idunnoman1222 wrote: post #265
Workaccount2 wrote: post #260

Earlier quoted context omitted.

I don't see this as something that would be hard to overcome. Sora for instance has already shown the ability for a diffusion model to maintain object permanence. Flux recently too has shown the ability to render the same person in many different poses or images.

Where does a sora video turn around backwards? I can’t maintain such consistency in my own dreams.

I don't know of an example (not to say it doesn't exist) but the problem is fundamentally the same as things moving out of sight/out of frame and coming back again.

Re: Diffusion models are real-time game engines

#298

Post by ajuc »

bamboozled wrote: post #270
dewarrn1 wrote: post #221

Earlier quoted context omitted.

Great observation. And not entirely unlike normal human visual perception which is notoriously vulnerable to missing highly salient information; I'm reminded of the "gorillas in our midst" work by Dan Simons and Christopher Chabris [0]. [0]: https://en.wikipedia.org/wiki/Inattentional_blindness#Invisi...

Are you saying if I turn around, I’ll be surprised at what I find ? I don’t feel like this is accurate at all.

The opposite - if you turn around and there's something that wasn't there the last time - you'll likely not notice if it's not out of place. You'll just assume it was there and you weren't paying attention.

We don't memorize things that the environment remembers for us if they aren't relevant for other reasons.

Re: Diffusion models are real-time game engines

#299

Post by meheleventyone »

GaggiX wrote: post #271
meheleventyone wrote: post #250

Earlier quoted context omitted.

I think you misunderstand me. I don't mean a snap evaluation and deciding between two very-short competing videos which is what the participants were doing. I mean doing an actual analysis of how well the simulation matches the ground truth of the game. What I'd posit is that it's not actually a very good replication of the game but very good a replicating short clips that almost look like the game and the short time…

>I mean doing an actual analysis of how well the simulation matches the ground truth of the game. Do you mean the PSNR and LPIPS metrics used in paper?

No, I think I've been pretty clear that I'm interested in how mechanically sound the simulation is. Also those measures are over an even shorter duration so even less relevant to how coherent it is at real game scales.

Re: Diffusion models are real-time game engines

#300

Post by TheRealPomax »

If by "game" you mean "literal hallucination" then yes. But if we're not trying to click-bait, then no: it's not really a game when there is no permanence or determinism to be found anywhere. It might be a "game-flavoured dream simulator", but it's absolutely not a game engine.
Post reply on HN