I have a feeling that Half Life 3 will have groundbreaking AI NPCs.
Importance of context management in AI NPCs
11–20 of 27 posts
Re: Importance of context management in AI NPCs
#12I think people forget that all AI inference is role playing to some extent. It pretends to be a chatbot, or a programmer, or whatever. There is no real difference between that and telling it to pretend to be a wizard.
Re: Importance of context management in AI NPCs
#13Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
The problem with AI NPCs is actually not strictly a context problem and cannot be fixed with prompt engineering or RAG, because the LLM knows a _vast_ amount of stuff outside of the context you feed it.
No matter how you tell it how to roleplay or how many instructions you give it or don't give it, there is always the problem that you can ask it to write a front end app in JS for you and it will. Or ask it about the theory of relativity or anything else that the AI is capable of conversing about but the character would not be. It is trivially easy to jailbreak out of fictional personas.
Re: Importance of context management in AI NPCs
#14My OS gets slower and buggy if I don't reboot. So I'll try to convince my users to reboot often and optimize their workflows for rebootability.
Feels like trying to solve a problem that shouldn't exist in the first place.
Once an OS that doesn't require reboots appear, this concept will look silly and everyone that optimized their workflows for reboots will look like dorks.
Re: Importance of context management in AI NPCs
#15Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
> Maybe a simplified model will run to represent time passing while we are away, in games where that sort of thing matters
More primitive, but this is technically how AI in Bethesda games work, has this background simulation for NPCs going on when out of sight. Think it's mainly focused on movement patterns though.
Dwarf fortress as well has it's world simulation setup such that the current tile is higher precision than the rest of the world. In fact I think plugging the output of this simulation would work perfectly as context for an LLM (or Embedding?) depending on what you wanted to accomplish...
Re: Importance of context management in AI NPCs
#16Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
will the fact that their lives pause while we’re gone shatter the illusion?
When two people leave the house in the morning and return home at night, does either person truly know what happened with the other person? Almost every interaction you have with other people is just “filling in” reality (reality is sparse, as far as you are concerned).
Person A: Saw a guy in a pikachu outfit at work today
Person B: Okay.
What else is there to say in reality? Just accept the context given to you.
Re: Importance of context management in AI NPCs
#17Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
In X4, the space game[1], they simulate the entire "universe" in the background. That includes ships flying around trading, fighting, stations running short of supplies etc etc.
To have this work on a modest computer, they have multiple fidelity levels, which primarily affects observable stuff. So near the player, all details of flight, collisions, projectiles etc are simulated. Further away certain collision checks are skipped and such.
Really far away, the simulation runs at a much reduced rate, flight simulation is significantly simplified, statistical methods are used for calculating weapon damage and such.
This does have the issue of discrepancies between levels. In the lowest fidelity mode, fleet A might consistently beat fleet B, while at the highest fidelity level it can be the opposite.
That said it's quite fun in the sense that playthroughs are seldom the same, and it allows for the player to make significant impact by simply helping one faction produce more goods that allows them to build more ships etc.
Re: Importance of context management in AI NPCs
#18Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
> Maybe a simplified model will run to represent time passing while we are away, in games where that sort of thing matters More primitive, but this is technically how AI in Bethesda games work, has this background simulation for NPCs going on when out of sight. Think it's mainly focused on movement patterns though. Dwarf fortress as well has it's world simulation setup such that the current tile is higher precision t…
Dwarf Fortress + LLM could be a really interesting idea. Although, I also wonder if it is like… so much effort was put into the simulation itself. Almost seems like a shame to start filling any gaps with LLM, haha.
Re: Importance of context management in AI NPCs
#19Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
Add the NPCs to a table to be ticked at some rate or triggered by some event. Calculate their actions and execute on them at that rate. Most NPCs do not need to be ticked. Some will need to be ticked every game day. Some can simply be ticked after some event game altering event.
Maybe I'm not understanding you but it doesn't seem all that different from running a non-AI story NPC.
Re: Importance of context management in AI NPCs
#20Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
The problem with AI NPCs is actually not strictly a context problem and cannot be fixed with prompt engineering or RAG, because the LLM knows a _vast_ amount of stuff outside of the context you feed it. No matter how you tell it how to roleplay or how many instructions you give it or don't give it, there is always the problem that you can ask it to write a front end app in JS for you and it will. Or ask it about the…
Unlike current videogames, LLMs by default flips the relationship with the limits: the agent is completely open-ended by default, but the player has to play along to a certain extent.
This makes the experience a lot closer to a tabletop game, where you can say that your D&D character does anything you want, and it's a negotiation between the player, the dungeon master, the dice, and the rules as to whether you allow it to happen and what the result is.
An LLM by default tends to be the world's most permissive dungeon master, so the burden of keeping things consistent shifts to the player. Early AI Dungeon gameplay is a typical example. Feels kind of like forum roleplaying, if you're familiar with it--there's no technical limit to what you can write so social conventions (and the mods) are what's preventing you from god-modding.
This is very different from the "try to break everything" way a lot of video game players approach things.
We might be able to eventually build an LLM system where fantasy knights don't know javascript and you can't summon a dragon by typing "there's a dragon." But that's going to take a lot of hard technical work, because it's very unnatural for an LLM out of the box.