Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Handbook.md shows that long policy documents do not reliably govern agents

arxiv.org

Post reply on HN
Print view View original

161–170 of 237 posts

Re: Handbook.md shows that long policy documents do not reliably govern agents

#161

Post by pixl97 »

jason1cho wrote: post #124

I don't understand why a model has to follow the instructions. Don't get surprised when it shows its true color. Plus, do users (not the researchers) really check whether the response follows the instructions?

Models don't have to follow instructions, but during RLHF that is one of the things they are scored on so a premise of the idea is part of the model, but it's also balanced on accomplishing the end goal. They model may determine, correctly or incorrectly, that your rules suck and do what it thinks is best.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#162

Post by joshmoody24 »

nzeid wrote: post #158
Aurornis wrote: post #115

Earlier quoted context omitted.

> Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. This is just not true. Any local LLM you can host on consumer-accessible hardware has all of these defects, too. Adjusting the knobs doesn’t solve everything. The closest you can get to frontier performance is Kimi K3, but you’re not hosting th…

This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear nor was it suggesting that local inference today is sufficient for all tasks. At the heart of it, self-hosting liberates your use cases from all the horribly opaque configuration, shadow prompting, etc. And local models are only getting better and more diverse every month.

> Want it to go away, almost like magic?

True, almost like magic and magic are not the same thing, but I'd hesitate to call this a strawman.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#163

Post by gwerbin »

DanHulton wrote: post #47
simpaticoder wrote: post #13

Earlier quoted context omitted.

>Want it to go away, almost like magic? Local inference. Ah yes, magic that costs the same as a new car.

Not necessarily! If you have a semi-recent MacBook with even 32GB, you can run 20GB models that are pretty damn smart, with room to spare for the rest of your toolchain. If you’re reasonably connected to the code you’re writing and prompting the AI at the level of the code, not the level of the feature, you can get some fantastic results. Sure, it’s not the completely automated dreamland that’s been sold, but it’s st…

I tried this on my work M4 with 48 GB of memory and it's not as useful as I had hoped.

If you set a large context window you end up using all your memory. Prefill is slow. And the 10-20B models are somewhat poor at understanding intent and tend to think in circles, so if you set a small thinking budget to control context size the model ends up answering based on some half formed thoughts.

Or something like batch document summarization or image classification or whatever yeah it's great. But for "agentic" and coding workflows I think you would need a completely different harness setup and prompting style compared to what we are used to from the big proprietary models. I certainly have not yet figured out what that is.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#164

Post by irthomasthomas »

8note wrote: post #120
wongarsu wrote: post #103

Earlier quoted context omitted.

LoRAs or even full fine-tunes would be much cheaper than that, and with some investment in the right infra could be updated regularly. And at least LoRAs can be swapped in and out cheaply, making them usable in large-scale inference providers. But there seems to be limited appetite in offering this. Both Anthropic and OpenAI no longer offer fine tuning for current models

does lora do a good job at teaching the model new things that werent in the training data? without trillions of examples of following instructions at a million context length, im not convinced the behaviour is in the weights to begin with

No. Lora is for tuning behaviour and how the model applies what it learned in training. Teaching a model new facts is still expensive.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#165

Post by john_strinlai »

nzeid wrote: post #158
Aurornis wrote: post #115

Earlier quoted context omitted.

> Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. This is just not true. Any local LLM you can host on consumer-accessible hardware has all of these defects, too. Adjusting the knobs doesn’t solve everything. The closest you can get to frontier performance is Kimi K3, but you’re not hosting th…

This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear nor was it suggesting that local inference today is sufficient for all tasks. At the heart of it, self-hosting liberates your use cases from all the horribly opaque configuration, shadow prompting, etc. And local models are only getting better and more diverse every month.

>The parent comment wasn't suggesting that the inherent defects magically disappear

that is almost exactly what they said though..?

"Want it to go away, almost like magic? Local inference. [...] all of the common LLM defects will go away.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#166

Post by solarkraft »

nzeid wrote: post #158
Aurornis wrote: post #115

Earlier quoted context omitted.

> Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. This is just not true. Any local LLM you can host on consumer-accessible hardware has all of these defects, too. Adjusting the knobs doesn’t solve everything. The closest you can get to frontier performance is Kimi K3, but you’re not hosting th…

This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear nor was it suggesting that local inference today is sufficient for all tasks. At the heart of it, self-hosting liberates your use cases from all the horribly opaque configuration, shadow prompting, etc. And local models are only getting better and more diverse every month.

I get your reading. My interpretation is that they’re claiming enshittification and that the local models don’t have that.

It’s an understandable view, but I’d be astonished by any local model processing very long context better than any frontier model (and now many racks are we talking).

Re: Handbook.md shows that long policy documents do not reliably govern agents

#167

Post by dk_sig11 »

honkycat wrote: post #68

This is what spec driven development tries to solve. Multiple rounds of generating small contacts documents that grow from the original idea , trying to keep each slice small enough to process for a human to approve/disprove . Eventually it leads to a long list of tasks grouped by functionality. You start a new context and the orchestrator agent dispatches tasks to sub agents with a limited amount of information prov…

I have not seen any real life examples of successful application of SDD. I heard a lot of marketing BS, but nothing real. My own experiments with SDD produced very disappointing results even on very small projects.

Re: Handbook.md shows that long policy documents do not reliably govern agents

#168

Post by Rumudiez »

msejas wrote: post #64
drob518 wrote: post #43

Earlier quoted context omitted.

What do you mean by a graph of one shot prompts?

Most people jump straight to agents when what they actually need is a graph. Example: a mining company receives free-text reports from field geologists. You could have: Geologist report -> LLM call extracts minerals we are looking for (you inject a db query result on the user prompt), locations, assay mentions and risks into structured fields -> LLM call classifies evidence into positive indicators, negative indicato…

I would call that a pipeline, which may be a type of graph, but might have communicated your point more concretely to begin with

Re: Handbook.md shows that long policy documents do not reliably govern agents

#169

Post by Muromec »

haus20xx wrote: post #85
drob518 wrote: post #56

Earlier quoted context omitted.

Doesn’t that blow your token cache hit rate and balloon your costs (essentially everything is billed at the input token rate, not the cached rate)?

If it moved a block like => //Removed project_prompts //reinserted Cache would break but if you did instead //Duplicated with new user messsage It wouldn't bust cache, it would just make your input prompts slightly larger. Technically inefficient as you're duplicating the same rules over and over but I imagine for a smallish checklist/principles that it is tremendously more efficient than a cache break every message

By the time it does 10 turns of tool calls it forgets about the nagging again. It can and does ignore direct instruction from the user prompt too, sometimes repeatedly

Re: Handbook.md shows that long policy documents do not reliably govern agents

#170

Post by fibuladev »

smu3l wrote: post #101
nonethewiser wrote: post #59

Earlier quoted context omitted.

This is not what the article is talking about. Its talking about policy documents not it forgetting something 5 prompts ago. In fact you adding things to CLAUDE.md is more what its talking about.

Conceptually the same thing though. Claude.md is sourced at the beginning of the session, so will be pretty far back in context, just like user prompts from the beginning of the session.

[flagged]
Post reply on HN