Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

351–360 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#351

Post by Aeolun »

majormajor wrote: post #267
Aeolun wrote: post #255

Earlier quoted context omitted.

There is only so much you can do with prompts. To go from the 70% accuracy you can achieve with that to the 95% accuracy I see in Claude Code, the context is absolutely the most important, and it’s visible how much effort goes into making sure Claude retrieves exactly the right context, often at the expense of speed.

Why are we drawing a difference between "prompt" and "context" exactly? The linked article is a bit of puffery that redefines a commonly-used term - "context" - to mean something different than what it's meant so far when we discuss "context windows." It seems to just be some puffery to generate new hype. When you play with the APIs the prompt/context all blurs together into just stuff that goes into the text fed to…

> Why are we drawing a difference between "prompt" and "context" exactly?

Because they’re different things? The prompt doesn’t dynamically change. The context changes all the time.

I’ll admit that you can just call it all ‘context’ or ‘prompt’ if you want, because it’s essentially a large chunk of text. But it’s convenient to be able to distinguish between the two so you can talk about the same thing.

Re: The new skill in AI is not prompting, it's context engineering

#352

Post by imiric »

zaptheimpaler wrote: post #187

I feel like this is incredibly obvious to anyone who's ever used an LLM or has any concept of how they work. It was equally obvious before this that the "skill" of prompt-engineering was a bunch of hacks that would quickly cease to matter. Basically they have the raw intelligence, you now have to give them the ability to get input and the ability to take actions as output and there's a lot of plumbing to make that ha…

That might be the case, but these tools are marketed as having close to superhuman intelligence, with the strong implication that AGI is right around the corner. It's obvious that engineering work is required to get them to perform certain tasks, which is what the agentic trend is about. What's not so obvious is the fact that getting them to generate correct output requires some special skills or tricks. If these tools were truly intelligent and capable of reasoning, surely they would be able to inform human users when they lack contextual information instead of confidently generating garbage, and their success rate would be greater than 35%[1].

The idea that fixing this is just a matter of providing better training and contextual data, more compute or plumbing, is deeply flawed.

[1]: https://www.theregister.com/2025/06/29/ai_agents_fail_a_lot/

Re: The new skill in AI is not prompting, it's context engineering

#353

Post by phyalow »

kazga wrote: post #344
phyalow wrote: post #342

Earlier quoted context omitted.

“non-deterministic machines“ Not correct. They are deterministic as long as a static seed is used.

That's not true in practice. Floating point arithmetic is not commutative due to rounding errors, and the parallel operations introduce non-determinisn even at temperature 0.

What? You can get consistent output on local models.

I can train large nets deterministically too (CUBLAS flags). What your saying isn't true in practice. Hell I can also go on the anthropic API right now and get verbatim static results.

Re: The new skill in AI is not prompting, it's context engineering

#354

Post by skydhash »

felipeerias wrote: post #234
JohnMakin wrote: post #10

> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…

If someone asked you about the usages of a particular element in a codebase, you would probably give a more accurate answer if you were able to use a code search tool rather than reading every source file from top to bottom. For that kind of tasks (and there are many of those!), I don't see why you would expect something fundamentally different in the case of LLMs.

But why not provide the search tool instead of being an imperfect interface between it and the person asking? The only reason for the latter is that you have more applied knowledge in the context and can use the tool better. For any other case, the answer should be “use this tool”.

Re: The new skill in AI is not prompting, it's context engineering

#355

Post by HarHarVeryFunny »

I guess "context engineering" is a more encompassing term than "prompt engineering", but at the end of the day it's the same thing - choosing the best LLM input (whether you call it context or a prompt) to elicit the response you are hoping for.

The concept of prompting - asking an Oracle a question - was always a bit limited since it means you're really leaning on the LLM itself - the trained weights - to provide all the context you didn't explicitly mention in the prompt, and relying on the LLM to be able to generate coherently based on the sliced and blended mix of StackOverflow and Reddit/etc it was trained on. If you are using an LLM for code generation then obviously you can expect a better result if you feed it the API docs you want it to use, your code base, your project documents, etc, etc (i.e "context engineering").

Another term that has recently been added to the LLM lexicon is "context rot", which is quite a useful concept. When you use the LLM to generate, it's output is of course appended to the initial input, and over extended bouts of attempted reasoning, with backtracking etc, the clarity of the context is going to suffer ("rot") and eventually the LLM will start to fail in GIGO fashion (garbage-in => garbage-out). Your best recourse at this point is to clear the context and start over.

Re: The new skill in AI is not prompting, it's context engineering

#356

Post by theasisa »

kachapopopow wrote: post #345

I'll quote myself since it seems oddly familiar: --- Forget AI "code", every single request will be processed BY AI! People aren't thinking far enough, why bother with programming at all when an AI can just do it? It's very narrow to think that we will even need these 'programmed' applications in the future. Who needs operating systems and all that when all of it can just be AI. In the future we don't even need hardw…

This reminds me of the talk The Birth And Death Of JavaScript, https://www.destroyallsoftware.com/talks/the-birth-and-death...

Re: The new skill in AI is not prompting, it's context engineering

#357

Post by Havoc »

Honestly this whole "context engineering" trend/phrase feels like something a Thought Leader on Linkedin came up with. With a sprinkling of crypto bro vibes on top.

Sure it matters on a technical level - as always garbage in garbage out holds true - but I can't take this "the art of the" stuff seriously.

Re: The new skill in AI is not prompting, it's context engineering

#358

Post by __loam »

Turskarama wrote: post #294
nonethewiser wrote: post #251

Earlier quoted context omitted.

>Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "sticks" in non-deterministic space. I dont quite follow. Prompts and contexts are different things. Sure, you can get thing into contexts with prompts but that doesn't mean they are entirely the same. You could have a long running conversation with a lot in the context. A given prompt may work poorly, w…

Both the context and the prompt are just part of the same input. To the model there is no difference, the only difference is the way the user feeds that input to the model. You could in theory feed the context into the model as one huge prompt.

Sometimes I wonder if LLM proponents even understand their own bullshit.

It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation?

They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.

Re: The new skill in AI is not prompting, it's context engineering

#359

Post by SonOfLilit »

shakna wrote: post #315
groestl wrote: post #282

Earlier quoted context omitted.

Well, to get the right thing into the context in a performant way when you dealing with a huge dataset is definitely engineering.

Engineering tends to mean "the application of scientific and mathematical principles to practical ends". I'm not sure there's much scientific or mathematical about guessing how a non-deterministic system will behave.

The moment you start building evaluation pipelines and running experiments to validate your ideas it stops being guessing

Re: The new skill in AI is not prompting, it's context engineering

#360

Post by __loam »

Aeolun wrote: post #351
majormajor wrote: post #267

Earlier quoted context omitted.

Why are we drawing a difference between "prompt" and "context" exactly? The linked article is a bit of puffery that redefines a commonly-used term - "context" - to mean something different than what it's meant so far when we discuss "context windows." It seems to just be some puffery to generate new hype. When you play with the APIs the prompt/context all blurs together into just stuff that goes into the text fed to…

> Why are we drawing a difference between "prompt" and "context" exactly? Because they’re different things? The prompt doesn’t dynamically change. The context changes all the time. I’ll admit that you can just call it all ‘context’ or ‘prompt’ if you want, because it’s essentially a large chunk of text. But it’s convenient to be able to distinguish between the two so you can talk about the same thing.

It's all the same blob of text in the api call
Post reply on HN