Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

Post reply on HN
Print view View original

341–350 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#341

Post by throwatdem12311 »

sennalen wrote: post #331
throwatdem12311 wrote: post #303

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

Why do you need another human to review the code of a human why can't these things get code right the first time

Why is human-like behavior the standard for comparison? These things should be better than us that’s the entire point.

They should not be making the same mistakes that humans make that’s the half the reason for building tools.

Re: The new rules of context engineering for Claude 5 generation models

#342

Post by throwatdem12311 »

rayiner wrote: post #336
throwatdem12311 wrote: post #303

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

This is like saying “why do I need a compiler when I can do better register allocation by hand in assembly?” Quick and “good enough” will win over “better.”

Compilers have been better than humans for decades. It’s also straight translation and not non-deterministic like LLMs are. LLMs are trained on the corpus of the entirety of these decades of research and implementations and yet they are still terrible at it.

They don’t randomly decide to drop your database either.

Re: The new rules of context engineering for Claude 5 generation models

#343

Post by andai »

the_gipsy wrote: post #286
andai wrote: post #260

Earlier quoted context omitted.

I'm not sure. I think I had a reasonable amount of metacognition as a child. I remember being eight years old, and having a really cool idea but forgetting it. So I devised a plan to retrieve it. I'm just going to rewind time. I'm going to go back to doing what I was doing when I had that idea, and then it'll come back to me. And so I remembered that I had been fiddling with my seat belt when I had the idea. And so I…

> So I devised a plan to retrieve it. I'm just going to rewind time. I'm going to go back to doing what I was doing when I had that idea, and then it'll come back to me. You narrated the ideas back. It's words and language. There is no hidden, unknown, layer of "thought".

>You narrated the ideas back

No, I rewound my visual memory, to see what I was doing, and then I did it again, to provide the same stimulus, to retrieve the lost memory.

I've met a few people who have nonverbal cognition, which is also not visual.

It's a bit like this, a "direct manipulation of ideas".

> The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought. The psychical entities which seem to serve as elements in thought are certain signs and more or less clear images which can be “voluntarily” reproduced and combined… The above-mentioned elements are, in my case, of visual and some of muscular type. Conventional words or other signs have to be sought for laboriously only in a secondary stage, when the mentioned associative play is sufficiently established and can be reproduced at will.

https://cognitivemedium.com/srs-mathematics

Though as I mentioned, I had this more as a child, and I've become overreliant on language, and this "direct" facility has starkly deteriorated. But as the author explains in the rest of this article, this fluency can be regained by sheer force of will, by simply working on a very narrow problem space obsessively for weeks at a time.

(We don't know if that works for everyone, or if it's something weird like perfect pitch. I suspect everyone could gain a great deal of fluency -- that the chunking would reach such a high order level as to make the mnaipulation feel transparent. But possibly, the types of chunks would be different depending on the person.

More data needed!)

Re: The new rules of context engineering for Claude 5 generation models

#344

Post by throwatdem12311 »

xpct wrote: post #326
throwatdem12311 wrote: post #303

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

Lately, I've been having less and less success using the models for work. Simple things like: 'write this in a separate file' (writes it in the same file) 'format this aiming for 5 LoC' (emits newline after every comma, resulting in 17 LoC) 'include 5 warmup steps before you measure runtime' (omits it completely and apologizes after I point it out) I wouldn't be as opposed to using the models if they weren't as unrel…

It doesn’t even follow simple directives in my Claude.md:

NEVER write comments unless explicitly directed to.

Still overly comments every gd helper function.

Re: The new rules of context engineering for Claude 5 generation models

#345

Post by Tadpole9181 »

throwatdem12311 wrote: post #303

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

I disagree, this is all because it's understood.

You need a system promot because an LLM is fundamentally a token predictor. It needs to be primed for the work it's going to do, otherwise it's next token prediction has too little to go off in the beginning and goes nuts.

You use a separate AI to review the code, because it's a token predictor trained mostly on accomplishing tasks in a cost effective way - we haven't made AGI here. The first shot that actually makes the code will have rationalizations in its memory as well as prior research, which bias token prediction to accepting that as true (remember how they had to train our sycophancy?). Then because of the desire to be cost effective, they're slightly lazy, and so it won't always do the research to find new edge cases and problems and missing tests the initial research didn't find.

So spin up a separate, clean slate, and ask for it to review from scratch. Or have that one spin up multiple smaller ones to have them specialize in specific concerns or domains (security, data model corruption, code quality), then have the orchestrator validate those concerns and stitch them into a cohesive response.

I can't help you on your last question. Saying they are degrading in quality is not even close to my experience. 5.6 and Opus/Fable 5 are have been a huge step up. Though I do need to adjust memory rules as these new models come out, since ground-up retrained models often come with their own quirks that replace old ones - causing old, specific memories to have unintended side effects.

Re: The new rules of context engineering for Claude 5 generation models

#346

Post by bensonperry »

throwatdem12311 wrote: post #341
sennalen wrote: post #331

Earlier quoted context omitted.

Why do you need another human to review the code of a human why can't these things get code right the first time

Why is human-like behavior the standard for comparison? These things should be better than us that’s the entire point. They should not be making the same mistakes that humans make that’s the half the reason for building tools.

i mean this is the same thing as self-driving cars and all sorts of technology - humans are our benchmark, being better than a human and cheaper immediately unlocks massive value everywhere. if i can hire a human-level programmer for ~$100 USD per month, that is an incredible upgrade from that costing me ~$10k per month!

Re: The new rules of context engineering for Claude 5 generation models

#347

Post by the_gipsy »

andai wrote: post #343
the_gipsy wrote: post #286

Earlier quoted context omitted.

> So I devised a plan to retrieve it. I'm just going to rewind time. I'm going to go back to doing what I was doing when I had that idea, and then it'll come back to me. You narrated the ideas back. It's words and language. There is no hidden, unknown, layer of "thought".

>You narrated the ideas back No, I rewound my visual memory, to see what I was doing, and then I did it again, to provide the same stimulus, to retrieve the lost memory. I've met a few people who have nonverbal cognition, which is also not visual. It's a bit like this, a "direct manipulation of ideas". > The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought.…

There are no movie clips in your brain that you replay. You are re-narrating the scene, you just may not notice it.

Re: The new rules of context engineering for Claude 5 generation models

#348

Post by SirHackalot »

throwatdem12311 wrote: post #303

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

> becoming disillusioned

> becoming?

It's all arcane, superstitious nonsense. Nobody actually knows how to work with these models is right. We've replaced software engineering with prompt astrology and Al whispering. The peak comedy of it all is watching people like Karpathy publish "skills" that read like psychoanalysis, just offering basic verbal instructions with a completely straight face...

Re: The new rules of context engineering for Claude 5 generation models

#349

Post by flawn »

firasd wrote: post #3

I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent. I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you remove…

I feel that. I thought I am the only one who has this issue, with Claude always adding this superfluous BS. I feel totally backwards when realizing I am trying to teach this model how to write good technical docs, while I could have already done it in the same time.

Anyone has experiences with other models? I feel like GPT is much more concise?

Re: The new rules of context engineering for Claude 5 generation models

#350

Post by fastball »

nullbio wrote: post #206
fastball wrote: post #157

Earlier quoted context omitted.

The fact you called it ChatGPT instead of Codex makes me suspicious of this entire comment.

Weird take. I meant to say GPT, not ChatGPT. Was referring to the model, not the harness.

Is it? There are shills and bots all over the internet, and referring to the model as ChatGPT is probably common for some people, but not for software engineers that actually track and compare the models (at least in my experience).
Post reply on HN