Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Handbook.md shows that long policy documents do not reliably govern agents

arxiv.org

Post reply on HN
Print view View original

131–140 of 237 posts

Re: Handbook.md shows that long policy documents do not reliably govern agents

#131

Post by Aurornis »

slopinthebag wrote: post #126
Aurornis wrote: post #115

Earlier quoted context omitted.

> Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. This is just not true. Any local LLM you can host on consumer-accessible hardware has all of these defects, too. Adjusting the knobs doesn’t solve everything. The closest you can get to frontier performance is Kimi K3, but you’re not hosting th…

Do you work for a frontier lab by any chance?

No.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#132

Post by dinfinity »

wongarsu wrote: post #26

Any model with a good score on this benchmark would have a good claim on superhuman abilities. Humans are pretty terrible at being thrown a long policy document and being expected to follow it And while we shouldn't anthropomorphize these models too much, I wouldn't be surprised if many of the core reasons for failures are similar. Working memory is a limited resource; you can only focus on so many things at once; re…

Exactly: The best way to deal with this for humans is to use procedural scripts for different tasks, referring to the relevant bits of the (declarative) documentation.

I would imagine that doing something similar (using agent skills for insurance) would work much better for AI.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#133

Post by DiabloD3 »

_fat_santa wrote: post #25
DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

At my org we've been building AI agents and one internal rule we have is to use at most 50% of the models context window with the recommendation to not go over 25% for large context window models. Anytime I see a "1M Context Window", my brain always goes "Gotcha so a 250k usable window"

Every single analysis I've seen done by anyone has basically come to the same conclusion: assuming sane BF16 or Q8 model quant, and BF16 or Q8 KV context quant (ie, not intentionally screwing over the model), you get about 250k before it pukes.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#134

Post by irthomasthomas »

Analemma_ wrote: post #107
tedggh wrote: post #99

Earlier quoted context omitted.

Why 25% and not 12% or 40%? Is this an arbitrary vibestimate or you had some tests done that pointed you to 25%? I am genuinely interested in how others deal with context issues. Also I would think the usable context window is variable depending on the task, for example summarizing documents vs analyzing large, scattered and complex instructions.

In my own experience (both Claude and Gemini 1M), you start running into problems around 60-70%. A 25% ceiling sounds like total cargo culting to me.

Benchmarks show a large drop in quality as context grows. Your opus will be acting like haiku above 300k tokens.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#135

Post by DiabloD3 »

simpaticoder wrote: post #13
DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

>Want it to go away, almost like magic? Local inference. Ah yes, magic that costs the same as a new car.

Tell me where I can buy a car for $1k or $2k.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#136

Post by DiabloD3 »

barumrho wrote: post #51
DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

Would you mind sharing some of your setup? Which model and which params do you tweak (e.g. temperature)?

See the end of https://news.ycombinator.com/item?id=49100144

Re: Handbook.md shows that long policy documents do not reliably govern agents

#137

Post by ux266478 »

TiccyRobby wrote: post #106
ux266478 wrote: post #87

Earlier quoted context omitted.

As parent implies, they're testing the wrong control mechanism. Why are you using policies instead of real controls over the weights and inference pipeline? Well the answer is that VC-backed companies decided AI is not a domain expert tool for highly competent technical users, it's a magic oracle for the lowest common denominator. So you don't get any of the actually useful controls, just context engineering like tha…

And what is the difference between cloud modeld vs local inference regarding this? It is still unclear in this thread - asking out of curiosity.

The ultimate difference isn't necessarily down to remote vs local. You can run a "local" model on the cloud the same way you could on a local machine and have all the same control over it. You can modify the weights, inject behavior, stack on arbitrary LoRAs, use whatever runtime you want, etc.

The problem arises in the case of anthropic, openai, etc. where you can't do those things. I think they have LoRA services that they'll charge you for, but that's about the extent of it. Think of it like having a compiler and its source code, and the operating system it runs on vs simply having access to invoking it via a web service.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#138

Post by solatic »

This is why the future is in deterministic static analysis and policy-as-code frameworks.

Policy-in-English? Model implicitly complains that it's TL-DR.

Ask the model to write code that checks your policy, then add that code behind a simple validation hook (e.g. "check your work by running 'just validate'") that the harness knows to always run after changes? It suddenly becomes the most law-abiding citizen ever.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#139

Post by qpwoeiruty1 »

msejas wrote: post #64
drob518 wrote: post #43

Earlier quoted context omitted.

What do you mean by a graph of one shot prompts?

Most people jump straight to agents when what they actually need is a graph. Example: a mining company receives free-text reports from field geologists. You could have: Geologist report -> LLM call extracts minerals we are looking for (you inject a db query result on the user prompt), locations, assay mentions and risks into structured fields -> LLM call classifies evidence into positive indicators, negative indicato…

This is a great point, and one that has been blowing my mind for a while. The public think of AI as this black box thing approaching human intelligence. They don't know that when you look under the hood of a lot of ai products you see a string of prompts that anyone could put together themselves. The companies are just putting them together in a workflow.
Post reply on HN