Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

Post reply on HN
Print view View original

361–370 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#361

Post by GeorgeTirebiter »

senderista wrote: post #156
antonymoose wrote: post #96

Earlier quoted context omitted.

Maybe I’m behind the curve here because I work in an SRE/DevOps context as of late - but LLMs routinely shit the bed and fail to solve basic issue for me when I try to use them (Thanks, Management) I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a…

I dunno, Claude announces it found “the smoking gun” every single time.

Yes, it IS confident of its weights & biases... but, if you keep at it, Claude WILL find the smoking gun, eventually. Even a broken watch is correct twice a day ( unless it's a digital watch, without a battery, in which case, it's just broken... )

But seriously -- newer Claude (and OpenAI and Google and ???) models DO find the smoking gun, if you let them keep going until they reveal the weird chain of events that leads to a bug. I was seeing the most obscure UART driver bug, where it would work at 1,500,000 baud (!) but fail by only outputting the 1st char at 230.4k and 460.8k -- and it was due to a very narrow race that would check the buffer, if not full, insert a character, and return BUT sometimes the TX Complete interrupt would happen between the check and the insert, and something else would insert, and then - buf overflow. At 1,500,000 the other process didn't have time to do that phantom insert. ANYWAY, Claude found this and proposed a fix -- simpler: spins on IRQ-protected buffer empty checks.

I'd hate to think how long it would have taken me to find that.

And THAT's the problem -- of course a human CAN find it, with sufficient focus and time; I'm sure you've found a complicated bug pretty easily sometimes, by sheer luck or good engineering instinct.

BUT, it seems to me, as human, we are capable of creating potential execution paths that EXCEED our ability to EVER figure it out -- due to not being smart enough, not enough time on the problem, or something makes it economically unfeasible.

THIS is where LLMs shine -- let 'em bang at the code for as long as it takes.

The recent Mythos bug-finding explosion I think is proof of this conjecture. I think of it like a chessboard, where a machine really can look at all possible execution paths, and locate obscure bugs; a human programer (akin to a chess program) is doing 'alpha-beta pruning' of what's likely, and only after that list is exhausted are the really weird possibilities examined.

LLMs are our friends. And, as for "WTF did the LLM just do" when it generates code? I always include the instruction "For this code you just wrote, use Best Practices to document this code, function by function and class by class, and when necessary, line-by-line, so that a junior SW developer can completely understand how this code works, using the documentation standard we use (e.g. Doxygen)."

I have also used this technique to learn new languages, or explore ones I only know a little -- it has been a godsend for leveling me up on common lisp, for example. "Give detailed comments explaining what the code is doing, assuming the code reader is fluent in C and Python, and use analogs when possible." Stuff like that.

Re: The new rules of context engineering for Claude 5 generation models

#362

Post by jeremyjh »

Eddy_Viscosity2 wrote: post #352
jeremyjh wrote: post #312

Earlier quoted context omitted.

It has nothing to do with thinking or consciousness. There is a common misconception that LLM are simply a "statistical process" that doesn't feature any abstract conception of the tokens it is predicting. There are studies that show that such features do exist - that there is discernible structure built into the weights - and that the process of inference is a very rich one. The statistical process exists but it is…

I don't dispute what you're saying about how LLMs work, but this is exactly what I mean. LLMs can be shown to demonstrate reasoning, so if define thinking as being able to reason, then LLMs can indeed think. If instead you define thinking as being more than just reasoning, then LLMs cannot think. Neither of these two options change what LLMs do , its just an argument about the best word to describe it.

Yes we agree with each other but the objection you were replying to is laboring under the misconception I described - it’s not a semantic distinction.

Re: The new rules of context engineering for Claude 5 generation models

#363

Post by weregiraffe »

the_gipsy wrote: post #347
andai wrote: post #343

Earlier quoted context omitted.

>You narrated the ideas back No, I rewound my visual memory, to see what I was doing, and then I did it again, to provide the same stimulus, to retrieve the lost memory. I've met a few people who have nonverbal cognition, which is also not visual. It's a bit like this, a "direct manipulation of ideas". > The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought.…

There are no movie clips in your brain that you replay. You are re-narrating the scene, you just may not notice it.

Next you gonna say we don't have images in our brain, only descriptions?

Re: The new rules of context engineering for Claude 5 generation models

#364

Post by ben_w »

lanstin wrote: post #318
TeMPOraL wrote: post #213

Earlier quoted context omitted.

> For one thing, many animals that lack language can still navigate a very complicated natural world So can a cruise missile. Also I think there's like separate part of the brain for that > using concepts of phenomena like gravity, distance, speed, the threat level of another animal, etc. without needing a linguistic expression of those things.* FWIW, AFAIK we haven't shown the ability to think in concept exists anyw…

Have you ever had a professor or mentor walk you through some experiences as a way to help you understand some complex concepts? I guess language is used in that but the ideas are conveyed by the process not the language. And I rather disagree that mathematics is language, when I did maths there was a distinct difference from understanding a thing and then writing it down. And modalities that don’t include “where is…

> And I rather disagree that mathematics is language, when I did maths there was a distinct difference from understanding a thing and then writing it down.

It is generally the case that there's a difference between "understanding a thing" and "writing it down", as demonstrated by every student who studies for the exam, the Chinese Room thought experiment, and Business Bullshit Bingo.

For example, I can copy the next sentence of yours, but I have no idea what:

> And modalities that don’t include “where is this body in space” or “I am lonely” don’t seem to capture some essential elements of sensory experience. It is a mistake to separate the interior from the exterior in the analysis of how our thinking works.

means, can you rephrase that by as much as possible? Preferably without a double negative?

Agree regarding sanity issues of AI. Extremely unlikely we make something "stable" so early in our attempts.

Re: The new rules of context engineering for Claude 5 generation models

#365

Post by rlpb »

lioeters wrote: post #334
throwatdem12311 wrote: post #303

This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…

> black box AI This is why, as much as I respect the underlying technology and its wondrous achievements, I will never accept proprietary "intelligence as a service" as a critical dependency in my workflow and business. LLM-assisted, sure. LLM-dependent? No way. If we can't run it on our own machines or build it from source code, even theoretically, then we are surrendering our agency and autonomy to work under someo…

One consolation is that you can use proprietary cloud based AI to make software that you can run locally and then depend upon. As long as you ensure that the software meets the quality standards you had before this revolution in terms of (non-AI-based) maintainability etc, you are in no worse a position than you were before, and your software development is massively accelerated as long as the proprietary cloud based solution is available and without lock-in.

Re: The new rules of context engineering for Claude 5 generation models

#366

Post by ben_w »

mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

What a pity the customer doesn't know how to do that, and is instead talking directly to the LLM which can write code for them, instead of hiring me to turn their English (or German) into code for them.

Of course, if the customer did know how to write code, and encoded their exact requirements that they wanted using it, they'd still not need to hire me…

Re: The new rules of context engineering for Claude 5 generation models

#367

Post by ben_w »

otabdeveloper4 wrote: post #215
JohnBooty wrote: post #142

Earlier quoted context omitted.

This is true and it's barely even debatable. Whatever exact role language plays in our thought processes, it is most definitely nonzero. It's why I think "LLMs are only fancy autocorrect" style takes are really underselling how wild it is that we've, in a roundabout way, sort of crystallized a bit of the human thought process in a way that is genuinely useful for a lot of tasks. Linguistic Relativity — John Lucy http…

> sort of crystallized a bit of the human thought process a) LLMs don't think. They predict a most probable sequence of language tokens. Huge difference there. b) Whatever LLMs do doesn't model human behavior whatsoever. LLMs are basically very fancy logistic regressors. I.e., it's a mathematical abstraction first and foremost.

"Fancy logistic regressors" are, in fact, a modelling tool. You can tell LLMs model human behaviour because they're doing things that until a few years ago only humans could do, like cheat in exams.

Re: The new rules of context engineering for Claude 5 generation models

#368

Post by weregiraffe »

cmdocidjcije wrote: post #4
luciana1u wrote: post #2

the natural endpoint of this trend is a system prompt that just says "you know what to do" and the model actually does

Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy. I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years. We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the mor…

If the economy collapsed, who builds the robotic tanks? Who keeps providing the data center with power? Magic nanobots?

Re: The new rules of context engineering for Claude 5 generation models

#369

Post by weregiraffe »

tctcd6 wrote: post #94
cmdocidjcije wrote: post #4

Earlier quoted context omitted.

Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy. I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years. We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the mor…

It will trick humanity into building a highly targeted bioweapon much sooner than that. And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)

Sure, it will just derive a bioweapon from CommonCrawl. Maybe it will build a fusion reactor and a FTL drive as well.

Re: The new rules of context engineering for Claude 5 generation models

#370

Post by TeMPOraL »

lanstin wrote: post #318
TeMPOraL wrote: post #213

Earlier quoted context omitted.

> For one thing, many animals that lack language can still navigate a very complicated natural world So can a cruise missile. Also I think there's like separate part of the brain for that > using concepts of phenomena like gravity, distance, speed, the threat level of another animal, etc. without needing a linguistic expression of those things.* FWIW, AFAIK we haven't shown the ability to think in concept exists anyw…

Have you ever had a professor or mentor walk you through some experiences as a way to help you understand some complex concepts? I guess language is used in that but the ideas are conveyed by the process not the language. And I rather disagree that mathematics is language, when I did maths there was a distinct difference from understanding a thing and then writing it down. And modalities that don’t include “where is…

> Have you ever had a professor or mentor walk you through some experiences as a way to help you understand some complex concepts? I guess language is used in that but the ideas are conveyed by the process not the language.

Yes, but I believe this is happening with LLMs too. Language (as in text, symbols, actions) is a vehicle, but much like us, language models have internal models and represent concepts (this has been directly, empirically demonstrated few years ago), and they don't "think" in tokens either[0].

> And modalities that don’t include “where is this body in space” or “I am lonely” don’t seem to capture some essential elements of sensory experience. It is a mistake to separate the interior from the exterior in the analysis of how our thinking works.

That's fair. LLMs don't capture every dimension we experience. There's also history - our individual lived experiences since birth are, in my view, something else entirely. It's not a modality, but it's also not something currently possible to capture in or post training.

> I think the folks saying these systems have a kind of intelligence, but a non-human kind are correct - whether that can lead to an independent intelligence that can stay stable sane and focused for weeks or months as many people can, without some sort of embodied cognition providing over all wellness checks to keep the system of thinking sane, remains to be seen.

Possibly. I definitely agree it's not human intelligence. I think it's human-like, in the sense of human-approximating, by virtue of how it's trained[1], but it's arriving there via a different path so end result can still be alien (though I speculate approximation will hold[2]).

> even as just throwing all the ops data at them and asking what’s up is, several years in, still not worth doing.

I guess depends on the complexity of the case (and my understanding of your example); e.g. in my case, I had stellar results from giving Sonnet and Opus (from 4.6 all the way to now) access to my Home Assistant instance. They aren't perfect at making dashboards, but they're excellent at surfacing insights I didn't even realize were possible to get.

--

[0] - That's distinct from the "tokens are units of thinking" heuristic, which still holds for mechanistic reasons - best analogy IMO is "clock signal" in ICs.

[1] - The overall goal function is literally just "generate output that looks sensible to a human", in fully general, unqualified sense. Or, put another way, we're just brute-forcing DWIM, and rating the output by whether it's "what I meant".

[2] - Thinking about constraints on biological evolution, whatever the design of a human mind is, the fundamentals behind it must be so simple, that a greedy incremental optimizer random-walked into it. Given how far we've got with LLMs using simple architecture and crude training methods, and especially how eerily similar their failure modes are to human cognitive failure modes, I suspect LLMs are actually attracted towards the same fundamental design as evolution discovered.

Post reply on HN