Why No AI Games?

At Disney, they had an immersive Star Wars VR experience that I don't think is there any more (and was extremely pricey for a session).

They tricked the senses by having physical objects you could touch for every space in the game environment, there was stuff like wind, you could feel the heat of lava radiating off the ground in some spots - and body packs that would jolt you if you got shot, and a physically held "blaster" with haptic feedback.

I was blown away at how good it was and how immersive it felt. But, you need an entirely custom experience and game room and as I said it was very expensive (probably for good reason).

I did a conferencr talk maybe three years ago on using AI for games.

Since the talk, code, audio,and imagery have progressed leaps and bounds, almost all of the points on active AI use haven't changed much.

You have more capable smaller models now that means you could at least run local models in games, but you still don't get to see what it is going to do before the user does. Developers are acutely aware of the climate. if a game has a fault because the AI does something unexpected, it's not just a bug, It's a news story.

It still cannot be realistically used for multiplayer games. The models are prone to adversarial attacks, which can harm a multiplayer experience.

I did point out at the time that solo sandbox games could benefit. An active AI can be empowering for the player, unchecked empowerment can be great in a solo game. It destroys a multiplayer game.

Then there's just a lack of training in some of the specific areas of games. You pretty much have to filter game state down to text for a llm, I don't yet know of any gameplay embedding model.

I have played around with a little tower defence to see how much an AI can work on gameplay, It essentially has a REPL interface to the game logic where it can take actions and advance the game a tick at a time. It does an ok job, but there's still not much understanding of time and urgency to work with an active environment

"It is entirely jarring if an AI NPC that says something that's not consistent with the game state, or changes the game state in a way that violates the constraint of the understood rules/boundaries of the game"

I played with the Mantella mod for Skyrim a few months back and one of the problems with LLMs is you can't keep them on topic. I even used a custom trained one just for Skyrim, but the problem was it still has vast real world knowledge it shouldn't. For instance I asked a town guard where I could find Taylor Swift, who said she might be down at the tavern playing music. While the conversation didn't super overtly break the theme, the guard "stayed in character" and didn't start gushing about specific songs of hers or something, he still "knew" who she was. Current-gen AIs can't be fenced in very well. And almost every game idea needs some sort of fencing in.

If you play along with the AI it's not bad but if you poke the edges the illusion breaks quickly. You can't prompt a current-gen AI to just "forget everything you shouldn't know because it doesn't fit in the game universe."

I expect future architectures probably will fix this and that will help a lot. But we don't have them yet.

One thing I've learned is, there's a ton of interesting stuff in the world. Super talented people. But, what surface and gets widespread recognition and adoption is often very much determined by luck, and these days by algorithms. It's very possible there's some really cool AI games out there already, and they just haven't been discovered.

The other aspect is that a lot of people in the AI community are completely hooked on the reward-seeking of 'new thing'. Every hour there's some new tool to try out that 'changes everything'. But, without grinding it out you might never create something interesting by constantly jumping to the next new thing. Because, technological capabilities are just one part of the story.

Third, what makes something 'fun' often requires tons of iteration and a lot of LLM-type tools still rely on snapshots of applications. They're not fully there yet in terms of what happens in between moments. There are lots of principles they can draw on from and get a good prototype quickly, but it again comes back to really just slogging through lots of iterations until you find something fun.

Finally, I think with every new medium you get a first wave that really just copies the medium that came before it. Early radio was a lot like theater. Early TV was a lot like radio. I think eventually we'll see people – probably younger people – who are fully into AI and will find interesting things about that medium to express in ideas, and we'll get some really new and interesting stuff as a result.

In analogy to suspension of disbelief I'd like to postulate suspension of genericity:

when I sink countless hours skimming text content in some The Elder Scrolls I'm not really convinced that much of it is a story worth consuming (my opinion of TES is lower than my number of gameplay hours might suggest), but I'm at least open to the idea that it might be. Thousands of players have paid good money for consuming those stories, do who knows, I might actually find something meaningful, perhaps something I'll one day connect with a real person about, so it's not necessarily a given that it's all just filler.

But when it's an LLM generating the story for me and only for me, there's no other possibility. It's a hamster wheel time sink and I can't even pretend that it's not.