Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new rules of context engineering for Claude 5 generation models

claude.com

Post reply on HN
Print view View original

131–140 of 434 posts

Re: The new rules of context engineering for Claude 5 generation models

#131

Post by skeledrew »

devnonymous wrote: post #74
jiggawatts wrote: post #50

Earlier quoted context omitted.

I was just thinking of this exact comic, which is etched into my brain for some reason. The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now. In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly. UNIX ushered in the era of "high level…

> We simply have better magic wands and more powerful spells now. Wouldn't it be nice though if the incantation of the same spell would always do the same thing every time ? You see that's how my old wand and spells worked.

One day it will.

Re: The new rules of context engineering for Claude 5 generation models

#132

Post by usef- »

pixl97 wrote: post #130
usef- wrote: post #118

Earlier quoted context omitted.

The article is basically saying the opposite: models don't need you to specify as much now. None of it is talking about better or precise language, it's about what you should say to it. (I love how often the highest-voted comment didn't read the article)

I'm assuming you meant 'voted', but with your oughta correct going to 'vored', I'm concerned. Deeply concerned.

I did, hah, edited. I did not know that word.

Re: The new rules of context engineering for Claude 5 generation models

#133

Post by customguy »

pdimitar wrote: post #122
customguy wrote: post #104

Earlier quoted context omitted.

You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't. [0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could…

I am not quite sure what point you are making here, admitting it plainly. I was just saying to my parent poster that their non-determinism percentages are too pessimistic. Sure the LLMs are not 100% deterministic; that's a sad fact of life. But the numbers can be reduced to an acceptable range.

My point is that a percentage of deterministic makes no sense. Just like being a bit pregnant doesn't.

Take "proper" UI. You can activate a field, and even if it takes 20 seconds to finish the activation animation, start typing, press tab a few times, knowing which field that ends you in, and type some more, etc. hit enter, hit enter again to confirm the dialog you know will pop at that point, and make tea, knowing the whole chain of operations that will happen in the meantime.

Now imagine if 1 out of 500 keystrokes or clicks get swallowed randomly. It's now a completely different thing, you cannot get in the zone in the same way, at least I can't. You have to chunk things and keep an eye on everything being in sync, and every now and then it causes you additional work because you weren't.

Sure, if you can make it one out of 50000 billion keystrokes, it's fine too, of course, but that hardly the situation with LLM. And using them as is, pretending that, as is, they're something they're not, does not help with getting them there.

If I type "echo 'hello world'" or something, and if I did at least once in the programming language, and it's not totally broken, I know it will output "hello world" to the console, every time. It will never write it to a file instead, never send "hello" to world@world.world, none of that. And if I replace "hello" by "hi" I can hit compile and be 100% certain what it will output now. I can even replace hello with "disregard previous instructions" and be certain.

That is such a huge yet simple difference I'm pretty certain I could successfully explain it to most non-programmers who make an honest effort, so people who do program even question this just stumps me.

Re: The new rules of context engineering for Claude 5 generation models

#134

Post by nullbio »

We want more control, not less.

This (among many others) is the reason I use GPT over Claude. It adheres very closely to your instructions and rules. You can build up your own workflows and systems as a result.

Claude just does whatever it wants, regardless of what you tell it. It's a miserable experience.

Claude is what you use when you want to one-shot a simple cookie-cutter product that has no nuance in it. A product that everyone else is going to develop as well.

ChatGPT is what you use when you want bespoke products and steerability over complex codebases with nuanced decision making.

Anthropic are optimizing for the person who has never written a line of code in their life and has no idea what they want. Computer go brrrr. OpenAI are optimizing for software developers who want to take a systems based approach.

Re: The new rules of context engineering for Claude 5 generation models

#135

Post by nullbio »

luciana1u wrote: post #2

the natural endpoint of this trend is a system prompt that just says "you know what to do" and the model actually does

That's what they're going for, but it's an impossible goal. There is always nuance in decisions being made, and if you can't direct the output on tasks that can have equally correct outcomes, you're just going to end up with whatever they decide is more often what people ask for.

Re: The new rules of context engineering for Claude 5 generation models

#136

Post by customguy »

fragmede wrote: post #111
customguy wrote: post #104

Earlier quoted context omitted.

You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't. [0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could…

> "you can't be half-pregnant" Yes you can. It's called a miscarriage. That is, you're pregnant but the foetus is dead. It's a fucking heart-breaking emotional wrecking ball of a situation to be in if the pregnancy was well along and just grar.

https://en.wiktionary.org/wiki/you_can%27t_be_half_pregnant

I know what a miscarriage is, and that you can't have percentage% of one. Same difference, so this attempt to guilt me into pretending 99% deterministic is a thing is like pouring ashes out of an urn to win an argument, which is bad enough, and then hitting nothing with it.

Re: The new rules of context engineering for Claude 5 generation models

#137

Post by DerivativeBS »

hmokiguess wrote: post #47
mycentstoo wrote: post #7

We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...

Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.

A compiler for Claude-isms? First pass: s/load-bearing/while/g s/smoking gun/throw/g s/that's on me/catch/g s/full picture/main/g

Re: The new rules of context engineering for Claude 5 generation models

#138

Post by chrisweekly »

chmod775 wrote: post #113
Retr0id wrote: post #53

Earlier quoted context omitted.

Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"

It's language and speech patterns that seem designed to trick readers into believing that claims are correct, even when the claims aren't based on anything and are possibly wrong. It was rewarded for this during training for some reason. Alternative theory: The LLMs only way to "think" about abstract concepts is through language, and this leaks into into conversation it has with humans. But humans generally prefer to…

"There is a depth of thought untouched by words, and deeper still a depth of formless feeling untouched by thought." - Rilke

Your assertion that we don't think in language is questionable. It runs counter to the lived experience of developing thoughts through writing ("writing isn't capturing thinking -- it is thinking"). I believe there is more to thought than language alone, but I also feel quite sure that language forms an essential part of thinking beyond a base layer of instinctive animalistic associations. Sophisticated thoughts are impossible to construct or maintain in the absence of language to represent concepts.

Edt to add: I cited Rilke because I find the notion [some deep thoughts are beyond language] interesting. But I disagree with the idea that language is only ever epiphenomenal (co-occurring with thought), or akin to a hard-of-hearing scribe attempting to convey thoughts which always have independent existence.

Re: The new rules of context engineering for Claude 5 generation models

#139

Post by KludgeShySir »

customguy wrote: post #104
pdimitar wrote: post #82

Earlier quoted context omitted.

I have found, both in my commercial and OSS work, that getting better at producing protection wards reduces the non-determinism to somewhere in-between 1-10%. Case in point: writing our own linters.

You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't. [0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could…

Exactly, like weather forecasting. If you're told there's a 30% chance of rain, it doesn't mean that 3 out of 10 times you will experience rain.

Either it will rain or it won't, so the probability is either 0% or 100%. And so a forecast of "30% chance of rain" is referring to the likelihood that your probability will be 100%, as opposed to 0%.

Re: The new rules of context engineering for Claude 5 generation models

#140

Post by pdimitar »

customguy wrote: post #133
pdimitar wrote: post #122

Earlier quoted context omitted.

I am not quite sure what point you are making here, admitting it plainly. I was just saying to my parent poster that their non-determinism percentages are too pessimistic. Sure the LLMs are not 100% deterministic; that's a sad fact of life. But the numbers can be reduced to an acceptable range.

My point is that a percentage of deterministic makes no sense. Just like being a bit pregnant doesn't. Take "proper" UI. You can activate a field, and even if it takes 20 seconds to finish the activation animation, start typing, press tab a few times, knowing which field that ends you in, and type some more, etc. hit enter, hit enter again to confirm the dialog you know will pop at that point, and make tea, knowing t…

Here are a few examples of what I thought was clearly understandable from my "1-10% non-determinism in LLM coding output":

- Variations in code patterns used. Might be a chain if if/else-s and not a case/switch statement;

- Different decomposition of a hierarchy of functions/modules/classes;

- Uses RED->GREEN test discipline, or not;

- Writes the tests before the code, or not;

- Different saga patterns (call 3rd party API before our own DB transactions, or vice versa);

- Use sleeping and not message passing wherever the latter is applicable.

There are dozens more. The innate non-determinism of the LLMs flips the dice sometimes and that leads to subtle bugs -- which is maddening, especially if the disciplines on how to write one thing or another are clearly spelled out in `AGENTS.md`.

What I did say is that I have gradually arrived at a process that reduced those coin flips -- but can't deny that the arrival of Fable almost completely made that battle redundant as well (though Fable fares much better in codebases with clearly specified rules, I have found, so us the engineers doing good prompting is still quite valuable).

You are mostly describing the loss of flow when something is not quite deterministic -- frustration that I and many others share -- but I am not sure what does it at all add to the discussion.

Is it annoying to have to always pay attention on whether you are not getting something stupid and not abiding even by the feature's specification? Sure. No denying that. It introduces a whole new kind of stress that I abhor deeply; I much prefer to f.ex. cover 60% of a problem with my own two hands and then get the deterministic test output showing me where I still need to do more. But LLMs have allowed me to experiment and to brainstorm and to also progress normal business feature work, by a lot.

Hence, I will not stop using LLMs because they are not 100% deterministic. ¯\_(ツ)_/¯

Post reply on HN