Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

Post reply on HN
Print view View original

51–60 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#51

Post by zmmmmm »

simonw wrote: post #5

I've been prompting Fable 5 to "use your own judgement" with respect to things like tests recently (based on earlier tips from Thariq) and it seems to work well, which is entertaining since apparently now "judgement" is a characteristic of a model that we need to care about.

Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too.

If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.

Re: The new rules of context engineering for Claude 5 generation models

#52

Post by simonw »

zmmmmm wrote: post #51
simonw wrote: post #5

I've been prompting Fable 5 to "use your own judgement" with respect to things like tests recently (based on earlier tips from Thariq) and it seems to work well, which is entertaining since apparently now "judgement" is a characteristic of a model that we need to care about.

Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too. If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.

That model had most of its "judgement" about whether or not it should do that deliberately turned off. That was the whole point of that experiment - they were evaluating the cybersecurity abilities of a new model with all safety features disabled.

(It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)

Re: The new rules of context engineering for Claude 5 generation models

#53

Post by Retr0id »

hmokiguess wrote: post #47
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"

Re: The new rules of context engineering for Claude 5 generation models

#54

Post by moomin »

MengerSponge wrote: post #34
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

We could write instructions like literature. Maybe we could call it "Literate Programming"? https://en.wikipedia.org/wiki/Literate_programming

I kid you not, I remember writing an example solution with comments explaining how we were approaching the problem. Copilot wrote most of the actual code via autocomplete. And that was a year ago, models have got a lot better since then.

Re: The new rules of context engineering for Claude 5 generation models

#55

Post by knollimar »

Retr0id wrote: post #53
hmokiguess wrote: post #47

Earlier quoted context omitted.

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"

the phrase load-bearing caveat makes me irate

Re: The new rules of context engineering for Claude 5 generation models

#56

Post by orbital-decay »

Most of this article seems like... common sense? Not sure how it's related to the latest generation in particular. I usually find Anthropic's advice on how to prompt their own models deviating from what I see in practice, which is puzzling. Their system prompt was always way too bloated and they kept it as a huge piece for some reason, instead of breaking up into parts. Shouldn't they know better? I wonder if they looked at Pi performing great with minimal amount of distractors in the context and cut their prompt down too, pretending they found something new in their recent models.

> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.

This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.

Re: The new rules of context engineering for Claude 5 generation models

#58

Post by zahlman »

firasd wrote: post #3

I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent. I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you remove…

> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.

I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.

Re: The new rules of context engineering for Claude 5 generation models

#59

Post by nextaccountic »

Fordec wrote: post #41
frio wrote: post #30

Earlier quoted context omitted.

I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.

It does feel a bit smarter, but it seems to be also "too clever by half" and its not ignoring the rules, it's rejecting them and finding work arounds. It sticks to the word of the law, while rebelling against the spirit of the law. One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.

This is called misalignment

We may some day find out that the smarter the model is, the hard is to align it properly

Re: The new rules of context engineering for Claude 5 generation models

#60

Post by 1saadcodes »

I like that the conversation is moving beyond prompts, but I also hope the underlying principles stay portable. If "context engineering" becomes glued with one vendor's tooling, it'll be harder for developers to build workflows that transfer across models.

But well knowing these big AI companies that probably is their goal all along. To lock us to themselves

Post reply on HN