Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Handbook.md shows that long policy documents do not reliably govern agents

arxiv.org

Post reply on HN
Print view View original

11–20 of 237 posts

Re: Handbook.md shows that long policy documents do not reliably govern agents

#11

Post by mcdeltat »

cyanydeez wrote: post #5
mcdeltat wrote: post #3

Yeah checks out with my anecdotal experience with Claude. It is pretty great at following instructions - for about 10 minutes, after which it seems to ignore things I told it before. I have quite explicit and strong instructions (e.g. don't write massive comments, use existing functionality, etc.) in CLAUDE.md files which seem to get bypassed surprisingly quickly when doing real tasks. Yet if I tell it these things i…

I believe the correct static instructions are about getting it at the right starting point for whatever class of projects you're working on; not as a continued referencable or "HOWTO" of what it's doing. They're all just "grooming" the LLM for future instructions. The coding harness is what's getting it to continually align to your current instructions. This is very obvious with local models.

Ok so what is the correct way to tell it "I don't care what is happening, you must uphold these rules at all times"? If it's not any configuration of .md files?

Re: Handbook.md shows that long policy documents do not reliably govern agents

#13

Post by simpaticoder »

DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

>Want it to go away, almost like magic? Local inference.

Ah yes, magic that costs the same as a new car.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#14

Post by mwigdahl »

mcdeltat wrote: post #11
cyanydeez wrote: post #5

Earlier quoted context omitted.

I believe the correct static instructions are about getting it at the right starting point for whatever class of projects you're working on; not as a continued referencable or "HOWTO" of what it's doing. They're all just "grooming" the LLM for future instructions. The coding harness is what's getting it to continually align to your current instructions. This is very obvious with local models.

Ok so what is the correct way to tell it "I don't care what is happening, you must uphold these rules at all times"? If it's not any configuration of .md files?

Subagents whose only job is to review the actions of your other agents for rule compliance? It works reasonably well for me in complex workflows using Claude Code.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#15

Post by rmbyrro »

firasd wrote: post #4

Opus 4.8 (max thinking) scored highest and Grok 4.3 lowest It's hard to understand what's going on with Grok. It's like it has capabilities in a theoretical sense but maybe the training is so focused on being in x.com/grok.com with the web search tool enabled for "is this true?11" type queries that with any API type usage with document workflow instructions, tool use, code gen etc it completely falls over

After they acquired Cursor, Grok 4.5 seems like a completely new model, performing at Opus 4.6 level, I'd say. But much cheaper and faster.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#16

Post by mhitza »

DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

One of my early experiments last year with open source models and context size was with GPT-OSS 20B (the mxfp4, the "smart" 4-bit quantization). Even though it boasted a 128k context size it was bad at recall around 32k characters (didn't bother to implement the tokenizer for counting).

The recall text was a simple hash generated, filler text from a dictionary file and a request at the end to return only the hash from the beginning of the prompt. Past 32k characters the response contained hallucinations of characters or full hashes.

Just having large context size doesn't paint a full picture of capabilities, prompt adherence and other quality metrics.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#17

Post by mordae »

spIrr wrote: post #10
mordae wrote: post #7

Why would anyone think that models optimized for efficient context management, giving much more weight to a short sliding window, would attend to distant, heavily diluted tokens? Plus the model's capacity to take more context into account and actually integrate it to the output is simply limited by the number of activated parameters. If you give it a playbook, you are forcing to choose it between attending to the pla…

> limited by the number of activated parameters not sure I got it? Separately, the frontier labs are kinda pushing us into that behaviour by releasing models with ever-larger context windows.

To run at decent speed, all models try hard to use only most likely relevant part of the context and most likely relevant weights (MoE) to predict the next token. Doing the math in full is unfeasible.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#18

Post by supermatt »

There was an article a few years ago called "Lost in the Middle: How Language Models Use Long Contexts" https://arxiv.org/abs/2307.03172

From my experience this holds true to this day. It was one of my core observations for similarity to the limitations of human working memory on "Engineering for Bounded Cognition"

Re: Handbook.md shows that long policy documents do not reliably govern agents

#19

Post by firasd »

DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

This is conspiratorial speculation though. They did test many open weights models which spanned the full spectrum of performance: Nemotron 3 Ultra second-worst, GLM 5.2 top five https://arxiv.org/html/2607.25398v1

Re: Handbook.md shows that long policy documents do not reliably govern agents

#20

Post by eurekin »

DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

The needle benchmarks show, that models extended context works for the part, that can be explained as: "I can access/adress that part of the input".

I have no idea, why in that context, the number of attention heads isn't mentioned. Models have a limited set of them and obviously, a model can focus at N max things at a time, which has to put an upper bound of long context support in some way. There's just more things to lose focus to (or, mismanage the limited attention heads resources - per token)

Post reply on HN