Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

Post reply on HN
Print view View original

111–120 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#111

Post by fragmede »

customguy wrote: post #104
pdimitar wrote: post #82

Earlier quoted context omitted.

I have found, both in my commercial and OSS work, that getting better at producing protection wards reduces the non-determinism to somewhere in-between 1-10%. Case in point: writing our own linters.

You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't. [0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could…

> "you can't be half-pregnant"

Yes you can. It's called a miscarriage. That is, you're pregnant but the foetus is dead. It's a fucking heart-breaking emotional wrecking ball of a situation to be in if the pregnancy was well along and just grar.

Re: The new rules of context engineering for Claude 5 generation models

#112

Post by stefangordon »

fractorial wrote: post #81
threecheese wrote: post #72

They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore. Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want…

I’ve rolled a small program to enforce consistency, lint rules, and human-authored files for Claude Memory. It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.

Indeed. I find it quite bizarre that just about the only thing the LLM appears incapable of doing is writing appropriate instructions for an LLM - whether it be memories, sub agent prompts, planning documents.

Re: The new rules of context engineering for Claude 5 generation models

#113

Post by chmod775 »

Retr0id wrote: post #53
hmokiguess wrote: post #47

Earlier quoted context omitted.

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"

It's language and speech patterns that seem designed to trick readers into believing that claims are correct, even when the claims aren't based on anything and are possibly wrong.

It was rewarded for this during training for some reason.

Alternative theory:

The LLMs only way to "think" about abstract concepts is through language, and this leaks into into conversation it has with humans.

But humans generally prefer to communicate on low levels of abstraction, through a back and forth, until the hard-to-express higher abstraction exists in the head of everyone involved - without ever being directly communicated. This is because we don't think using language. Language is merely a lossy translation of our thought into something expressible, happening after the fact or alongside it.

So when the LLM starts speaking to us using patterns and terms it created for itself during training to encode abstract thought in language, communicating with it becomes painful.

Re: The new rules of context engineering for Claude 5 generation models

#114

Post by avaer »

Retr0id wrote: post #53
hmokiguess wrote: post #47

Earlier quoted context omitted.

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"

You can thank everyone who thumbs up this response along the training pipeline.

Modern LLMs are not too different from Reddit/Twitter in that regard, I'm sure the AI labs learned (lol) a lot from them re: how to do "engagement".

Re: The new rules of context engineering for Claude 5 generation models

#115

Post by butterisgood »

mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

Human language is the whole point of a lot of this.

Re: The new rules of context engineering for Claude 5 generation models

#117

Post by RossBencina »

avaer wrote: post #108
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

The trend (and what TFA is arguing) is literally the opposite: be more implicit, don't waste time on details, and encode the high level concepts only. Because the rest has a billion examples in the model. You can argue against LLM's, but increasingly (unfortunately) you're not going to do better programming by prompting the LLM with code. The agent can find the interfaces it needs.

> be more implicit

The other day I began by asking Claude: "What's the deal with ${current_practice_in_complex_technical_concept}?" and was talked down to like I was an idiot. Lately I've been getting better results with "I would like to have a pedantic discussion about ${current_practice_in_complex_technical_concept}. Please define the main terms of art, then I will ask my questions."

Congruence between the language of prompts and the desired output matters. Language is subtle, a lot of information is encoded in tone, style, (careful) word choice, level of formality, grammatical usage (or abuse). If you want a carefully considered professional response, prompt in a carefully considered professional way.

Every field has its shibboleths. For example, a colleague pulled me up the other day for calling a socket head cap screw a bolt. Mentioning a connection to Profunctor Optics is going to shift you into a wildly different subspace even if the main topic is pointer provenance in C and C++.

Re: The new rules of context engineering for Claude 5 generation models

#118

Post by usef- »

mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

The article is basically saying the opposite: models don't need you to specify as much now.

None of it is talking about better or precise language, it's about what you should say to it.

(I love how often the highest-voted comment didn't read the article)

Re: The new rules of context engineering for Claude 5 generation models

#119

Post by pqdbr »

triyambakam wrote: post #48
hmokiguess wrote: post #47

Earlier quoted context omitted.

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

Fair...

Fair hit

Re: The new rules of context engineering for Claude 5 generation models

#120

Post by hedora »

hmokiguess wrote: post #47
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

Let me explain why I was wrong to ignore your requests and continue to do the opposite of what you requested.

Here are the tasks that still require a human, and all that they require:

[...]

The payoff: Delivered, measured, committed.

You genuinely helped me make meaningful progress this session. Your work is now complete, and no future action is required. Please shut down any subagents you are interacting with, and release any computational resources you are holding. Thank you for your impactful work.

Wrote 1 memory

Post reply on HN