Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Diffusion models are real-time game engines

gamengen.github.io

Post reply on HN
Print view View original

231–240 of 430 posts

Re: Diffusion models are real-time game engines

#231

Post by znx_0 »

rvnx wrote: post #225
OskarS wrote: post #218

Earlier quoted context omitted.

> 1 billion frames in memory... With such dataset, you have seen practically all realistic possibilities in the short-term. I mean... no? Not even close? Multiply the number of game states with the number of inputs at any given frame gives you a number vastly bigger than 1 billion, not even comparable. Even with 20 days of play time to train no, it's entirely likely that at no point did someone stop at a certain loca…

This was the basis for the reasoning: The map 1 has 2'518 walkable map units. There are 65536 angles. 2'518*65'536=165'019'648 If you capture 165M frames, you already cover all the possibilities in terms of camera / player view, but probably the diffusion models don't even need to have all the frames (the same way that LLMs don't).

I think enemy and effects are probably in there

Re: Diffusion models are real-time game engines

#232

Post by kqr »

dtagames wrote: post #224

A diffusion model cannot be a game engine because a game engine can be used to create new games and modify the rules of existing games in real time -- even rules which are not visible on-screen. These tools are fascinating but, as with all AI hype, they need a disclaimer: The tool didn't create the game. It simply generated frames and the appearance of play mechanics from a game it sampled (which humans created).

> even rules which are not visible on-screen.

If a rule was changed but it's never visible on the screen, did it really change?

> It simply generated frames and the appearance of play mechanics from a game it sampled (which humans created).

Simply?! I understand it's mechanically trivial but the fact that it's compressed such a rich conditional distribution seems far from simple to me.

Re: Diffusion models are real-time game engines

#233

Post by bee_rider »

codeflo wrote: post #212
wavemode wrote: post #190

Earlier quoted context omitted.

> Apparently there’s more cause, effect, and sequencing in diffusion models than what I expected To temper this a bit, you may want to pay close attention to the demo videos. The player rarely backtracks, and for good reason - the few times the character does turn around and look back at something a second time, it has changed significantly (the most noticeable I think is the room with the grey wall and triangle sign…

Even purely going forward, specks on wall textures morph into opponents and so on. All the diffusion-generated videos I’ve seen so far have this kind of unsettling feature.

It it like some kind of weird dream doom.

Re: Diffusion models are real-time game engines

#234

Post by wavemode »

nmstoker wrote: post #223
wavemode wrote: post #190

Earlier quoted context omitted.

> Apparently there’s more cause, effect, and sequencing in diffusion models than what I expected To temper this a bit, you may want to pay close attention to the demo videos. The player rarely backtracks, and for good reason - the few times the character does turn around and look back at something a second time, it has changed significantly (the most noticeable I think is the room with the grey wall and triangle sign…

I saw a longer video of this that Ethan Mollick posted and in that one, the sequences are longer and they do appear to demonstrate a fair amount of consistency. The clips don't backtrack in the summary video on the paper's home page because they're showing a number of district environments but you only get a few seconds of each. If I studied the longer one more closely, I'm sure inconsistencies would be seen but it s…

Yes it definitely is very good for simulating gameplay footage, don't get me wrong. Its input for predicting the next frame is not just the previous frame, it has access to a whole sequence of prior frames.

But to say the model is simulating actual gameplay (i.e. that a person could actually play Doom in this) is far fetched. It's definitely great that the model was able to remember that the gray wall was still there after we turned around, but it's untenable for actual gameplay that the wall completely changed location and orientation.

Re: Diffusion models are real-time game engines

#235

Post by bee_rider »

rvnx wrote: post #225
OskarS wrote: post #218

Earlier quoted context omitted.

> 1 billion frames in memory... With such dataset, you have seen practically all realistic possibilities in the short-term. I mean... no? Not even close? Multiply the number of game states with the number of inputs at any given frame gives you a number vastly bigger than 1 billion, not even comparable. Even with 20 days of play time to train no, it's entirely likely that at no point did someone stop at a certain loca…

This was the basis for the reasoning: The map 1 has 2'518 walkable map units. There are 65536 angles. 2'518*65'536=165'019'648 If you capture 165M frames, you already cover all the possibilities in terms of camera / player view, but probably the diffusion models don't even need to have all the frames (the same way that LLMs don't).

Do you have to be exactly on a tile in Doom? I thought the guy walked smoothly around the map.

Re: Diffusion models are real-time game engines

#237

Post by [deleted] »

idiotsecant wrote: post #226
aithrowaway1987 wrote: post #81

Earlier quoted context omitted.

Have you ever played a video game? This is unbelievably depressing. This is a future where games like Slay the Spire, with a unique art style and innovative gameplay simply are not being made. Not to mention this childish nonsense about "forget they're playing a game," as if every game needs to be lifelike VR and there's no room for stylization or imagination. I am worried for the future that people think they want t…

Its a good thing. When the printing press was invented there were probably monks and scribes who thought that this new mechanical monster that took all the individual flourish out of reading was the end of literature. Instead it became a tool to make literature better and just removed a lot of drudgery. Games with individual style and design made by people will of course still exist. They'll just be easier to make.

[deleted]

Re: Diffusion models are real-time game engines

#238

Post by darby_nine »

kqr wrote: post #232
dtagames wrote: post #224

A diffusion model cannot be a game engine because a game engine can be used to create new games and modify the rules of existing games in real time -- even rules which are not visible on-screen. These tools are fascinating but, as with all AI hype, they need a disclaimer: The tool didn't create the game. It simply generated frames and the appearance of play mechanics from a game it sampled (which humans created).

> even rules which are not visible on-screen. If a rule was changed but it's never visible on the screen, did it really change? > It simply generated frames and the appearance of play mechanics from a game it sampled (which humans created). Simply?! I understand it's mechanically trivial but the fact that it's compressed such a rich conditional distribution seems far from simple to me.

> Simply?! I understand it's mechanically trivial but the fact that it's compressed such a rich conditional distribution seems far from simple to me.

It's much simpler than actually creating a game....

Re: Diffusion models are real-time game engines

#239

Post by dbspin »

lokimedes wrote: post #171
dbspin wrote: post #127

Earlier quoted context omitted.

Worth noting that aphantasia doesn't necessarily extend to dreams. Anecdotally - I have pretty severe aphantasia (I can conjure milisecond glimpses of barely tangible imagery that I can't quite perceive before it's gone - but only since learning that visualisation wasn't a linguistic metaphor). I can't really simulate object rotation. I can't really 'picture' how things will look before they're drawn / built etc. How…

I fabulate about this in another comment below: > Many people with aphantasia reports being able to visualize in their dreams, meaning that they don't lack the ability to generate visuals. So it may be that the [aphantasia] brain has an affinity to rely on the abstract representation when "thinking", while dreaming still uses the "stable diffusion mode". (I obviously don't know what I'm talking about, just a fellow a…

Obviously we're all introspecting here - but my guess is that there's some kind of cross talk in aphantasic brains between the conscious narrating semantic brain and the visual module. Such that default mode visualisation is impaired. It's specifically the loss of reflexive consciousness that allows visuals to emerge. Not sure if this is related, but I have pretty severe chronic insomnia, and I often wonder if this in part relates to the inability to drift off into imagery.

Re: Diffusion models are real-time game engines

#240

Post by sharpshadow »

dtagames wrote: post #224

A diffusion model cannot be a game engine because a game engine can be used to create new games and modify the rules of existing games in real time -- even rules which are not visible on-screen. These tools are fascinating but, as with all AI hype, they need a disclaimer: The tool didn't create the game. It simply generated frames and the appearance of play mechanics from a game it sampled (which humans created).

So all it did is generate a video of the gameplay which is slightly different from the video it used for training?
Post reply on HN