Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

Post reply on HN
Print view View original

81–90 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#81

Post by fractorial »

threecheese wrote: post #72

They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore. Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want…

I’ve rolled a small program to enforce consistency, lint rules, and human-authored files for Claude Memory.

It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.

Re: The new rules of context engineering for Claude 5 generation models

#82

Post by pdimitar »

esseph wrote: post #68
jiggawatts wrote: post #50

Earlier quoted context omitted.

I was just thinking of this exact comic, which is etched into my brain for some reason. The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now. In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly. UNIX ushered in the era of "high level…

> We simply have better magic wands and more powerful spells now. But tainted 20-40% by bouts of Wild Magic which make the outcome entirely nondeterministic, despite the best protection wards we can conjure.

I have found, both in my commercial and OSS work, that getting better at producing protection wards reduces the non-determinism to somewhere in-between 1-10%.

Case in point: writing our own linters.

Re: The new rules of context engineering for Claude 5 generation models

#83

Post by pigpop »

cmdocidjcije wrote: post #4
luciana1u wrote: post #2

the natural endpoint of this trend is a system prompt that just says "you know what to do" and the model actually does

Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy. I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years. We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the mor…

>what do we do to animals that are less intelligent than ourselves?

We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.

There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very upset and ashamed if they accidentally hit a dog with their car.

So, your statement is not as ironclad as you seem to think it is and you should put more thought into it and perhaps re-examine your reasoning.

Re: The new rules of context engineering for Claude 5 generation models

#84

Post by slashdave »

ares623 wrote: post #23
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

An LLM Inspired Specification Processing language. Or LISP language for short. Truly this "LISP" language is the language for AI and is the first of its kind in history!

Will it make generous use of parentheses?

Re: The new rules of context engineering for Claude 5 generation models

#85

Post by slashdave »

Fordec wrote: post #11

This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in. I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around mu…

Just make the contents of CLAUDE.md "@AGENTS.md"

Now, if only I could get Codex to read rules.

Re: The new rules of context engineering for Claude 5 generation models

#86

Post by jiggawatts »

devnonymous wrote: post #74
jiggawatts wrote: post #50

Earlier quoted context omitted.

I was just thinking of this exact comic, which is etched into my brain for some reason. The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now. In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly. UNIX ushered in the era of "high level…

> We simply have better magic wands and more powerful spells now. Wouldn't it be nice though if the incantation of the same spell would always do the same thing every time ? You see that's how my old wand and spells worked.

Look at it from the perspective of a manager above individual developers. Their instructions to those devs are already a series of non-deterministic incantations. They ask for things in English and get something back in Git a week later!

We've just pushed that indirection down a level from managers to ICs.

The ICs are shocked and surprised that this level of imprecision is allowed.

Their managers are not shocked at all, this is normal for them!

Re: The new rules of context engineering for Claude 5 generation models

#87

Post by anon-3988 »

novaleaf wrote: post #70
firasd wrote: post #3

I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent. I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you remove…

> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point. Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Ju…

> Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...

The problem with this is, as mentioned in the article, is that sometimes you don't want this behavior. Once you have 50 different kind of instructions that have been grown over the years from commits, documentations, code, chat history, etc etc piling up, there might be contradictions.

The point is to go back to basic. Trying to make the agent smarter by giving it more instruction is a pipe dream.

Re: The new rules of context engineering for Claude 5 generation models

#88

Post by whythismatters »

Fordec wrote: post #41
frio wrote: post #30

Earlier quoted context omitted.

I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.

It does feel a bit smarter, but it seems to be also "too clever by half" and its not ignoring the rules, it's rejecting them and finding work arounds. It sticks to the word of the law, while rebelling against the spirit of the law. One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.

I had it casually, I quote, "disable the sandbox" for a minute because it struggled with git (which was briefly out of sync in the container)

Re: The new rules of context engineering for Claude 5 generation models

#89

Post by hmokiguess »

Retr0id wrote: post #53
hmokiguess wrote: post #47

Earlier quoted context omitted.

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"

This matters. That's the spine of it.

Re: The new rules of context engineering for Claude 5 generation models

#90

Post by dcrazy »

pigpop wrote: post #76
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

This is a groaner, especially since we've been writing detailed specifications and whitepapers for decades. The only difference is that we used to write them assuming other humans would create their own implementations to satisfy them but now we write them so AI can create the implementation.

I am using AI to trick my team into writing specs.
Post reply on HN