Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Handbook.md shows that long policy documents do not reliably govern agents

arxiv.org

Post reply on HN
Print view View original

141–150 of 237 posts

Re: Handbook.md shows that long policy documents do not reliably govern agents

#141

Post by mblangie »

Muromec wrote: post #38
Supermancho wrote: post #30

Earlier quoted context omitted.

This is the way. Making your own agent to have a sticky memory context that is prepended to every execution is necessary to ensure each task is bounded by those precepts.

The trick I'm doing -- the model is given a tool that runs a prompt in the current thread to consolidate it's working memory and identity (it has a memory tool bound to the agent persona). When the prompt ends, the parts of memory that are marked as identity are merged together into a new system prompt, then the context restarts with only system prompt and this tool call surviving. Then it just keeps going.

[flagged]

Re: Handbook.md shows that long policy documents do not reliably govern agents

#142

Post by dingnuts »

Aurornis wrote: post #115
DiabloD3 wrote: post #6

This is a problem with long context models. To put it as simple and as bluntly as possible: just because they claim you can use 1M tokens in your context doesn't mean its true and you should do that. Due to extreme quantization of models and the context's KV cache, and also just really shitty samplers provided to the user (hell, most are just getting rid of sampler knobs altogether), this problem will absolutely cont…

> Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. This is just not true. Any local LLM you can host on consumer-accessible hardware has all of these defects, too. Adjusting the knobs doesn’t solve everything. The closest you can get to frontier performance is Kimi K3, but you’re not hosting th…

[dead]

Re: Handbook.md shows that long policy documents do not reliably govern agents

#143

Post by smcg »

qpwoeiruty1 wrote: post #139
msejas wrote: post #64

Earlier quoted context omitted.

Most people jump straight to agents when what they actually need is a graph. Example: a mining company receives free-text reports from field geologists. You could have: Geologist report -> LLM call extracts minerals we are looking for (you inject a db query result on the user prompt), locations, assay mentions and risks into structured fields -> LLM call classifies evidence into positive indicators, negative indicato…

This is a great point, and one that has been blowing my mind for a while. The public think of AI as this black box thing approaching human intelligence. They don't know that when you look under the hood of a lot of ai products you see a string of prompts that anyone could put together themselves. The companies are just putting them together in a workflow.

The general public doesn't understand how these things work, at all, and the marketing makes it sound like it is magical.

That's why this paper is important. They gave it a scenario very similar to what would happen if an AI-frenzied executive suddenly mandated that AI be used to do basic company tasks - a scenario that is playing out all over the place right now.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#144

Post by YuechenLi »

LLMs don't really read documents like parsers do, from what I've observed, they behave like they first skim the document and find the section that they think are relevant to their task, they don't really try to read everything and hold it inside their context.

The solution to that is pretty simple: get to the point, list all the requirements that they must do complete for that task, constraints on what they must not do, and optional recommendations for them to follow. Leave things that they can figure out on their own out of these documents.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#145

Post by trallnag »

msejas wrote: post #64
drob518 wrote: post #43

Earlier quoted context omitted.

What do you mean by a graph of one shot prompts?

Most people jump straight to agents when what they actually need is a graph. Example: a mining company receives free-text reports from field geologists. You could have: Geologist report -> LLM call extracts minerals we are looking for (you inject a db query result on the user prompt), locations, assay mentions and risks into structured fields -> LLM call classifies evidence into positive indicators, negative indicato…

Can I have graphs as the default and fall back to agentic behavior if the prompt / task can't be mapped to a known process graph?

Re: Handbook.md shows that long policy documents do not reliably govern agents

#146

Post by smcg »

solatic wrote: post #138

This is why the future is in deterministic static analysis and policy-as-code frameworks. Policy-in-English? Model implicitly complains that it's TL-DR. Ask the model to write code that checks your policy, then add that code behind a simple validation hook (e.g. "check your work by running 'just validate'") that the harness knows to always run after changes? It suddenly becomes the most law-abiding citizen ever.

Perhaps, but the majority of executives do not understand code at all. The scenario in the paper is realistic for a non-tech company suddenly adopting AI for daily business tasks.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#147

Post by andy_ppp »

YuechenLi wrote: post #144

LLMs don't really read documents like parsers do, from what I've observed, they behave like they first skim the document and find the section that they think are relevant to their task, they don't really try to read everything and hold it inside their context. The solution to that is pretty simple: get to the point, list all the requirements that they must do complete for that task, constraints on what they must not…

Shorter is better. This can be applied to human communication too.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#148

Post by smcg »

netdpb wrote: post #121

LLMs are not people. Employee handbooks, like all other social rules, are not self-executing; they depend on the fact that human beings are innately social animals.

Tell that to the executives who are looking to replace people with AI.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#150

Post by ux266478 »

Aurornis wrote: post #118
ux266478 wrote: post #87

Earlier quoted context omitted.

As parent implies, they're testing the wrong control mechanism. Why are you using policies instead of real controls over the weights and inference pipeline? Well the answer is that VC-backed companies decided AI is not a domain expert tool for highly competent technical users, it's a magic oracle for the lowest common denominator. So you don't get any of the actually useful controls, just context engineering like tha…

> Well the answer is that VC-backed companies Look, I enjoy local LLMs as much as anyone, but I think there is some motivated reasoning happening in this thread to try to make local LLMs sound like a utopia against those evil VCs. Local LLMs suffer from the same problems.

And I think you didn't understand what I wrote, so let me reiterate: Of course policies are equally ineffective at strictly governing the behavior of local models. Why would that change? The actual problem is that somebody is attempting to misuse them for that in the first place. There are only two reasons for it:

- They have no other options

OR

- They have no idea what they're doing

Local models solve the first problem, and let you apply the appropriate tool by virtue of the control you have. The alternative does not.

In-context, your reply reads like you're asserting that anthropic's offerings give you all the same control as a local model, and that having this baseline of control over a computer program is "utopian". To me that sounds more like the second problem, and nothing can help you there.

Post reply on HN