Makes me curious. What context is sent? Whole story, including later chapters? Just up to that paragraph? Planning and character description documents? Any way to control all of that?
I like to be strategic summarize the correct places when the context gets full.
There is a request inspector that shows everything that is sent.
I'm experimenting with prompt ordering strategies as well. this is surprisingly complext when you take into account the different model providers.
For example local models usually do a different form of prompt caching than for example openai and anthropic