Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Handbook.md shows that long policy documents do not reliably govern agents

arxiv.org

Post reply on HN
Print view View original

151–160 of 237 posts

Re: Handbook.md shows that long policy documents do not reliably govern agents

#151

Post by DoctorOetker »

wongarsu wrote: post #26

Any model with a good score on this benchmark would have a good claim on superhuman abilities. Humans are pretty terrible at being thrown a long policy document and being expected to follow it And while we shouldn't anthropomorphize these models too much, I wouldn't be surprised if many of the core reasons for failures are similar. Working memory is a limited resource; you can only focus on so many things at once; re…

"You would never hand a newbie a 124 page policy document and expect them to correctly apply it on the first task, or to do it reliably in the first month"

So the behavior policy should reside in the weights, not in some ever expanding KV cache context, like sticking it in the wrong orifice if the goal is to be reproductive.

Instead of putting the policy document in some tightly crammed RAM, shove it in the weights that already exist, online / post training.

It does make one wonder if there is some way to compute the change to the weights from the computed context (so that we can free it up) without essentially continued "pre" training RMAD after each communication round.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#152

Post by pixl97 »

kridsdale1 wrote: post #100
batshit_beaver wrote: post #83

Earlier quoted context omitted.

The challenge with comparing these things to humans, is that humans learn. A newbie might not respect your organization’s set of policies on day one, but what about 3 months in? Or 3 years? Meanwhile there’s still no reasonable mechanism for automatically fine tuning LLMs or adjusting their harnesses to make them better at completing your organization’s objectives more successfully. They’re still overwhelmingly gover…

Models learn. It just costs $10B and 1 year to do what a human does every night.

At least these costs are currently preventing the planets surface from being covered in paperclip maximizers for the moment.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#153

Post by leetrout »

ghostly_s wrote: post #39
leetrout wrote: post #2

HANDBOOK.md is a benchmark for long-context agentic instruction following, modeled on how enterprise employees follow company handbooks in their day-to-day work. Each task is a unique RL environment with internal tools and external MCP servers, spanning five enterprise domains: Finance, Medical Billing, Insurance, Logistics, and HR. The prompts reflect the actual jobs enterprise workers perform every day. Each task d…

Please don't paste walls of text into the comment field without quotation marks. It wastes all of our time.

It was copy/paste from my phone and when I posted there was no context / other comments and the github link was buried in the footer of the PDF of the paper.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#154

Post by pixl97 »

AnimalMuppet wrote: post #89
loremium wrote: post #78

Earlier quoted context omitted.

isn't it because there are too many contradictions and ambiguity? the reason it works for humans is because we don't apply everything at once either.

No, it's more than that. We can't remember everything. I hired, say, two years ago; as part of my onboarding process I had to read a bunch of policy and procedure documents, which were full of stuff that I didn't understand because I wasn't really in the context yet. So at the time, to me, those documents were full of arbitrary text that I didn't really understand. Some of it was rules that I had to follow, but at th…

Social effects are a huge part of being human. Quite often we just do what the people around us are doing. In general it's not a bad rule, but can quickly lead go mass casualty events when the crowd has no idea what's going on, or we're messing with dangerous things.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#155

Post by YuechenLi »

andy_ppp wrote: post #147
YuechenLi wrote: post #144

LLMs don't really read documents like parsers do, from what I've observed, they behave like they first skim the document and find the section that they think are relevant to their task, they don't really try to read everything and hold it inside their context. The solution to that is pretty simple: get to the point, list all the requirements that they must do complete for that task, constraints on what they must not…

Shorter is better. This can be applied to human communication too.

Not always. If an instruction is short and vague, then LLMs tend to fill in the blanks by guessing and the results are unpredictable. Semantic density is the key, prompts/instructions should be actionable and unambiguous over being terse and vague.

So, it's more about being concise.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#157

Post by Catloafdev »

thaumasiotes wrote: post #127
Catloafdev wrote: post #113

Earlier quoted context omitted.

There quite literally are internals that are called attention heads in modern LLMs. Saying 'there are no attention heads' is objectively false, which definition you follow is irrelevant.

Imagine the following conversation: How was your trip to Los Angeles? There were no movie stars! Is the second person's statement "objectively false"? Or is it more likely that you, Catloafdev, enjoy intentionally misinterpreting simple statements? You might notice that in the remainder of the sentence you quoted, hungryhobbit takes the trouble to define the attention heads that don't exist.

[dead]

Re: Handbook.md shows that long policy documents do not reliably govern agents

#158

Post by nzeid »

Aurornis wrote: post #115
DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

> Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. This is just not true. Any local LLM you can host on consumer-accessible hardware has all of these defects, too. Adjusting the knobs doesn’t solve everything. The closest you can get to frontier performance is Kimi K3, but you’re not hosting th…

This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear nor was it suggesting that local inference today is sufficient for all tasks.

At the heart of it, self-hosting liberates your use cases from all the horribly opaque configuration, shadow prompting, etc. And local models are only getting better and more diverse every month.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#159

Post by pixl97 »

missmoss wrote: post #116

This is real. I was very angry that AI kept breaking the rules I wrote, so I asked Claude to crawl its own history logs. And then I found: every time after it breaks a rule, the chance of breaking rules goes up. I feel this is like few-shot in reverse. Few-shot is supposed to be good examples AI should follow. But when it breaks a rule, we correct it, it keeps breaking, and this whole thing actually raises the probab…

>. But when it breaks a rule, we correct it, it keeps breaking, and this whole thing actually raises the probability of more violations.

In Pre-LLM days the 'nearest unblocked neighborhood' problem, where patching out one issue just immediately runs into another issue, or a different path back to the same issue. Since the models can learn new long time behaviors it's difficult to change the behavior without changing the context quite a bit.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#160

Post by bob1029 »

> Failures follow consistent patterns: agents let a plausible in-environment request override the standing policy

I prefer this behavior in a lot of situations. A plausible request can also be a genuine one.

Alignment at the grain of each tool response is way more effective than a static system prompt. I try to keep mine under 500 tokens. Why bother fighting recency bias? It's pretty much what you want most of the time.

Post reply on HN