Importance of context management in AI NPCs
walterfreedom.com
Importance of context management in AI NPCs
1–10 of 27 posts
Re: Importance of context management in AI NPCs
#2I have been hearing the term 'context engineering' and wanted to share my article about it, which is mostly focusing on AI npcs and not enterprise use cases.
Re: Importance of context management in AI NPCs
#3this guy really turned his note taking app project to a new look on context engineering
Re: Importance of context management in AI NPCs
#4For longer threads I find frequent prompting for sectional summaries along the lines of spaced repetition helpful.
Recently was evaluating some investment property for a friend that had valuable timber, historic notability, distressed foundation and structural elements, compromised access, and utility hookup challenges.
I would document and tackle all of the structural challenges then have ai summarize. Move on to all the access challenges. Summarize. All of the historic noteworthiness criteria etc.
Most people passed on the site because of one or other deal breakers but by synthesizing summaries was able to discover assistance programs targeted for “ag + historic” or “historic + access” initiatives etc that made the numbers pencil out as a “no-brainer” investment. These wouldn’t have unlocked otherwise without five decades of experience rehabilitating farmhouses in this part of the country.
Anyway, I imagine a similar approach would work for enriching NPC interactions in a game world.
Re: Importance of context management in AI NPCs
#5Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea.
Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time passing while we are away, in games where that sort of thing matters. This is fine because our NPCs are basically static, if you freeze them, and then wake them up when the PC shows up. They aren’t deep enough for the missing day to day events to matter.
But, with more complex NPCs, will the fact that their lives pause while we’re gone shatter the illusion? It seems like his original system (the universe broadcasts to every NPC even while they are not doing anything) could fudge that a bit and retain a feeling of ongoing background life, in some cases. While in the new system they are ready frozen…
I dunno. What to do? Maybe run a simplified model and have it generate some appropriate local events for the NPCs while they are frozen (some, fewer than when they were on the receiving end of the whole universe).
Re: Importance of context management in AI NPCs
#6Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
Parallelization and off-load to beefy computers. Run a more complete simulation, stream the results back to the player, and define boundaries where things become sequential.
EDIT: Also observation and action masking is being explored as a core part of agent design. Definitely a skill and something that needs to be thoughtful for it to work but see where action masking is being applied in PettingZoo environments using Langchain: https://pettingzoo.farama.org/tutorials/langchain/langchain/. I'm using something similar for a WW2 roguelike I'm working on. The idea is we train agents to operate as soldiers, squads, platoons, companies... With some abstractions and we can represent full fronts in WW2, battles with 1000s of agents, all in a cool ASCII environment (:
Re: Importance of context management in AI NPCs
#7Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
I think you can mostly fake this by waiting until the player reenters the range to generate what happened since the last time they interacted. If it's a complex simulation it won't work without more effort, but if it's flavor text like "Bob told me last week you killed the dragon, nice work!" then it can be done like 5ms after the player enters the simulation radius of the NPC.
Re: Importance of context management in AI NPCs
#8Sounds like his original system was for the AI characters to be fed all events that happen in the universe. Then he changed it so that the AI characters were only fed events they reasonably should have gotten. Makes sense and seems like a great idea. Something I wonder—often in a game, the focus follows the player character, and the universe stops when we go away. Maybe a simplified model will run to represent time p…
You can effectively accomplish something like this already simply through good writing and programming, accounting for these gaps as part of an NPC's characterization. It's a cool way to play with player expectations, and one of the core bases behind Toby Fox's games.
Re: Importance of context management in AI NPCs
#9>In my local ai (mistral-nemo) around 10 thousand tokens of context decreases my token gen speed from 70t/s to 20 t/s . And the LLM starts ignoring the context after a while.
as much as it pains me to say this, only cloud models are somewhat viable for this. AI-powered NPCs are my dream too, and after many attempts with countless local and cloud models, I've given up for now. locals are retarded and incurably sloppy, clouds can be tard-wrangled into producing somewhat decent prose, but they are prohibitively expensive.
mistral models are particularly soulless and full of cliches.
https://eqbench.com/creative_writing_longform.html
https://eqbench.com/results/creative-writing-longform/mistra...
Re: Importance of context management in AI NPCs
#10I have been hearing the term 'context engineering' and wanted to share my article about it, which is mostly focusing on AI npcs and not enterprise use cases.
Yes, common issue.
We will probably get something more formalized , like "context occlusion", for games in the future.