Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

Post reply on HN
Print view View original

91–100 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#91

Post by solenoid0937 »

pdantix wrote: post #65
orbital-decay wrote: post #56

Most of this article seems like... common sense? Not sure how it's related to the latest generation in particular. I usually find Anthropic's advice on how to prompt their own models deviating from what I see in practice, which is puzzling. Their system prompt was always way too bloated and they kept it as a huge piece for some reason, instead of breaking up into parts. Shouldn't they know better? I wonder if they lo…

> Most of this article seems like... common sense? i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days

I have a coworker that was complaining about Opus 5 and had random shitty skills and custom plugins wired in from YouTube tutorials watched over the past year. He also speaks with the model like it's GPT 4o.

Needless to say, none of the new models have worked well for him, and he refuses to remove the "tweaks" or update his style of communication, which is obviously breaking the experience.

Re: The new rules of context engineering for Claude 5 generation models

#92

Post by duxup »

firasd wrote: post #3

I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent. I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you remove…

Agreed I just find my own dance and adjust as I go.

Sometimes like a verbose coworker who just is that way… fine Claude, you be that way now. Things seem to change here and there anyway and sweating the small stuff of having to repeat myself, that’s ok.

Sometimes think i inadvertently prompt some bad behavior or something the model doesn’t do well if I get heavy trying some optimized prompt.

Re: The new rules of context engineering for Claude 5 generation models

#93

Post by kloud »

This is quite puzzling. Back in the day I saw the breakthrough with Opus 4.5, and since then was very lightweight on CLAUDE.md, trying to declare intent, rather than prescribe imperative instructions. If it was off the mark, I would see clear mismatch, which gave me opportunity to correct, usually to clear and strip context.

Starting with Fable 5, if it goes off the rails, it is more difficult to correct it, because it is overall wrong, but covers its tracks with plausible sounding arguments, so it is hard to pin point and correct.

Now, this article points out techniques that were useful to rely on models more, but those peaked at 4.6. Now according to benchmarks Opus 5 is on the frontier. But when when it has looser reigns, it ends up gaslighting me even more with abstract word soup than any model before.

Re: The new rules of context engineering for Claude 5 generation models

#94

Post by tctcd6 »

cmdocidjcije wrote: post #4
luciana1u wrote: post #2

the natural endpoint of this trend is a system prompt that just says "you know what to do" and the model actually does

Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy. I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years. We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the mor…

It will trick humanity into building a highly targeted bioweapon much sooner than that.

And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)

Re: The new rules of context engineering for Claude 5 generation models

#95

Post by nemo1618 »

mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

I am quite tired of this take, frankly. The implication is that if we continue iterating on prompt optimization, we're going to reinvent what, JavaScript? BASIC? Lisp?

English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling in the gaps" -- that is, extrapolating the expressed intent to cover all the little details that were left unspecified. This system works because humans are at least minimally competent at predicting the preferences of other humans. If your prediction turns out to be wrong, you get feedback and iterate.

Well, guess what. LLMs are also competent at predicting the preferences of humans. LLMs can "fill in the gaps" like no one's business. LLMs can iterate on requirements like no one's business.

Product managers do not speak to programmers in a language that encodes exact requirements, and yet working software somehow gets shipped anyway. LLMs do not need exact requirements either.

Re: The new rules of context engineering for Claude 5 generation models

#96

Post by antonymoose »

nemo1618 wrote: post #95
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

I am quite tired of this take, frankly. The implication is that if we continue iterating on prompt optimization, we're going to reinvent what, JavaScript? BASIC? Lisp? English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling…

Maybe I’m behind the curve here because I work in an SRE/DevOps context as of late - but LLMs routinely shit the bed and fail to solve basic issue for me when I try to use them (Thanks, Management)

I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a PM and Engineering Manager I wouldn’t have the job I have.

Re: The new rules of context engineering for Claude 5 generation models

#97

Post by Exoristos »

vatsachak wrote: post #26
grim_io wrote: post #22

Earlier quoted context omitted.

The success of LLM's (by usage) tells us that programming languages are still too close to the machine than the actual problem domain as defined by humans. If we truly had the right abstractions, no one would care to use LLM's for programming.

I disagree. I speak in code to the LLMs. It's just that LLMs are really good at reinventing the wheel that you were supposed to in your codebase. Recent example. struct TensorView { body: Arc , shape: [usize], stride: [usize], offset: usize, } Okay now fill in all the helper methods. And GPT 5.6 Sol did a good job.

The lengths some of y'all will go to to avoid using JetBrains products.

Re: The new rules of context engineering for Claude 5 generation models

#98

Post by vinnymac »

frio wrote: post #30
Fordec wrote: post #11

This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in. I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around mu…

I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.

I once described Mythos and then Fable as escaping the box we put them in. This would certainly fit that narrative.

Re: The new rules of context engineering for Claude 5 generation models

#99

Post by geuis »

ValentineC wrote: post #71
Fordec wrote: post #11

This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in. I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around mu…

I haven't been impressed with Opus 5 over the past ~30 hours either. It's made countless careless mistakes folding in plan amendments after they get reviewed by Sol, and has produced sloppy mockups (e.g. buttons overflowing past cards) despite all the supposed verification claims.

I finally made progress today on a deeply intractable problem that Sonnet was just incapable of solving despite hours of experimentation and multiple attempts. I swapped over to Opus in a new handoff session with the expectation of using at least one 5 hour session. It wasn't quick and took several hours but eventually Opus dialed in on several cascading base failures resulting from how the base vision model was being converted to coreml. It eventually was able to provide a combination of conversion and Swift changes that semi fixed the issue but still didn't solve the core conversion issues.

Then since Anthropic was so kind to provide $100 for Fable credits, I did another handoff to let Fable attack the root issues again. Several more hours and I'm back to seeing the same issues again and Fable is wandering around trying different things. I'm down about $40 of free money and still don't have a solution.

This is where having the human engineer in the loop benefits from a deep understanding of the problem domain. In this case, I don't yet.

I know the high level architecture I'm building, but the deep specifics of how vision models work and how conversion across platforms should be done isn't something I know yet.

So I'm left learning as I go and relying on constant feedback with the models to provide what guidance I can while learning exactly what is being done.

Someone who already knows these architectures would likely be able to get to the solution much faster.

Post reply on HN