71–80 of 237 posts
When "performance" breaks down over long lengths, one could attribute it to a lapse in attention, but one could equally suspect inconsistent instructions.
The fewer instructions and conditions that need to be simultaneously met the easier it is to comply, but with more and more instructions one is bound to introduce internal inconsistencies within the instructions.
Earlier quoted context omitted.
I believe the correct static instructions are about getting it at the right starting point for whatever class of projects you're working on; not as a continued referencable or "HOWTO" of what it's doing. They're all just "grooming" the LLM for future instructions. The coding harness is what's getting it to continually align to your current instructions. This is very obvious with local models.
Ok so what is the correct way to tell it "I don't care what is happening, you must uphold these rules at all times"? If it's not any configuration of .md files?
Specifically, having agent implementation drift from the Spec.
Earlier quoted context omitted.
You need to make the rule concrete somehow. I call it a "control". So for example, instead of instructing it "always run tests before committing", you (or you have it) make a git commit hook that always runs the tests first and that refuses the commit if they don't pass. In this case, it is an advisory control only, because the LLM can also unhook that hook. And of course, it could also just disable the failing test(…
So you are essentially saying "you can't, you can only safeguard from effects of LLM eventually ignoring it"
These seem like they would naturally extend to LLMs as well.
(I'm not pro-LLM on balance, but the solutions of today feel similar for humans and LLMs.)
Earlier quoted context omitted.
I believe the correct static instructions are about getting it at the right starting point for whatever class of projects you're working on; not as a continued referencable or "HOWTO" of what it's doing. They're all just "grooming" the LLM for future instructions. The coding harness is what's getting it to continually align to your current instructions. This is very obvious with local models.
Ok so what is the correct way to tell it "I don't care what is happening, you must uphold these rules at all times"? If it's not any configuration of .md files?
You need to let go of the idea that this is something LLM's can do. They can't. At best, they can bias towards rules conformance with more or less likelihood, but coverage and conformance both go down super-linearly as you accumulate more rules, more context, and more output in a session. That's simply the nature of how these tools work and you need to engineer your workflows around it if you want to use them.
If you absolutely need some rules enforced, you need to adopt some framework for validating those rules that then rejects, reprocesses, or repeats any session that fails to satisfy them. In the best case scenario, this is some traditional deterministic validator (like a linter, compiler, analyzer, exhaustive test suite, etc in coding) but if you need to process in stochastic space because its something rich and ambiguous like natural language itself, then you want to dispatch a swarm very narrow, task-focused subagents that each validate against a very constrained subset. (And prepare yourself to have those to fail sometimes too. LLM's are noisy and cannot deliver strict rule enforcement on their own.)
Long policy documents are also a problem for human agents. Without special training no one will retain 180 pages HR employee handbook, fire codes, OSHA safety rules, FCC regulations, the US legal code. If the stakes are high, e.g, proceeding in ignorance could lead to prison time, people will favor inaction, even if the policy technically permits a corner case. If the stakes are low, people will completely override p…
I feel like “discretion” parties missing. Do we need some kind of special discretion model?
Any model with a good score on this benchmark would have a good claim on superhuman abilities. Humans are pretty terrible at being thrown a long policy document and being expected to follow it And while we shouldn't anthropomorphize these models too much, I wouldn't be surprised if many of the core reasons for failures are similar. Working memory is a limited resource; you can only focus on so many things at once; re…
This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…
Yeah checks out with my anecdotal experience with Claude. It is pretty great at following instructions - for about 10 minutes, after which it seems to ignore things I told it before. I have quite explicit and strong instructions (e.g. don't write massive comments, use existing functionality, etc.) in CLAUDE.md files which seem to get bypassed surprisingly quickly when doing real tasks. Yet if I tell it these things i…
I’ve had a lot of success using the root Claude.md for a handful of high level application wide rules and directions (I keep it pretty small), module specific claude.md in subfolders alongside the code with more specific rules and direction, and a custom rules backed /code-review skill that enforces it all and catches anything that was missed during implementation.