These companies all wax on about how important context engineering is yet not one of them has released acceptable tooling for end users to visualize and understand the context window as it grows and shrinks during a session. Best Claude code can do? Warn you when you hit 80% full
Effective context engineering for AI agents
11–20 of 36 posts
Re: Effective context engineering for AI agents
#12I think any meaningful context engineering strategies will be trade secrets.
Maybe, but we'll be getting to a place where each LLM call gets cheaper, faster and has a larger context, it may not matter long term.
Context is often not the only issue. Really the issue is attention - context is a factor in how well the LLM handles attention to the broad scope of a task, but one can anecdotally easily observe the thing forget or go off the rails when only a fraction of the context window is being used. Oftentimes it’s effective to just say “don’t ever go above 20% of the max”
Re: Effective context engineering for AI agents
#13These companies all wax on about how important context engineering is yet not one of them has released acceptable tooling for end users to visualize and understand the context window as it grows and shrinks during a session. Best Claude code can do? Warn you when you hit 80% full
try /context in Claude Code
Re: Effective context engineering for AI agents
#14[dead]
Re: Effective context engineering for AI agents
#15Earlier quoted context omitted.
It's pretty straightforward, different optimizers have different requirements. Some require example inputs/outputs, others will just optimize on whatever you've got. You can use codex/claude code to set it up in order to bootstrap quickly, they're decent at it.
Does dspy support structured outputs?
Yes using signatures with types
Re: Effective context engineering for AI agents
#16It’s kind of useful but I suppose they just admit that failure rate increases with large context windows. My guess is that what happened to the presentation of those Meta glasses where the model would not do what was asked for.
Another interesting thought might be that long horizon tasks need different tooling, and with the shift to long running tasks you can use cheaper models as well. None of the big providers have good tools for that at the moment, so the only thing they can say is: to fix our contexts but still use their models.
Re: Effective context engineering for AI agents
#17I find you can give it a task and the full context in your 1st message, and also include (a) asking what files are needed to understand and complete task, and (b) ask if there’s anything ambiguous about the task/question. Then, when you get the response, create a new chat with just the files it recommends, and the ambiguities explained in the 1st comment. Sometimes you need a couple of rounds of this.
The you will have a good starting point, with less chance of running out of space before solving the task.
If you can’t give it full context at the beginning, you can give it a tree listing of the files involved, and maybe a couple of READMEs (if there are any) and ask it see if it can work out what files are needed, giving it a couple of files at a time, at its suggestion.
Re: Effective context engineering for AI agents
#18The article doesn't really give helpful advice here, but please don't vibe this. Create evals from previous issues and current tests. Use DSPy on prompts. Create hypotheses for the value of different context packs, and run an eval matrix to see what actually works and what doesn't. Instrument your agents with Otel and stratify failure cases to understand where your agents are breaking.
Otel meaning open Telemetry? Do they have special capability for tracking agents?
Re: Effective context engineering for AI agents
#19The article doesn't really give helpful advice here, but please don't vibe this. Create evals from previous issues and current tests. Use DSPy on prompts. Create hypotheses for the value of different context packs, and run an eval matrix to see what actually works and what doesn't. Instrument your agents with Otel and stratify failure cases to understand where your agents are breaking.
Otel meaning open Telemetry? Do they have special capability for tracking agents?
Yes, there is an otel standard for agent traces. You can instrument agents that don't natively support Otel via bifrost.
Re: Effective context engineering for AI agents
#20I think "output engineering" is equally as important, and steering with grammar (structured output with json schema or CFGs directly) is a huge win there I find:
https://platform.openai.com/docs/guides/function-calling#con...