Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

381–390 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#381

Post by skydhash »

__loam wrote: post #364
skydhash wrote: post #354

Earlier quoted context omitted.

But why not provide the search tool instead of being an imperfect interface between it and the person asking? The only reason for the latter is that you have more applied knowledge in the context and can use the tool better. For any other case, the answer should be “use this tool”.

The uninformed would rather have a natural language interface rather than learn how to actually use the tools.

The reason for the expert in this case (an uninformed that wants to solve a problem) is that the expert can use metaphors as a bridge for understanding. Just like in most companies, there's the business world (which is heterogeneous) and the software engineering world. A huge part of software engineer's time is spent translating concepts across the two. And the most difficult part of that is asking questions and knowing which question to ask as natural language is so ambiguous.

Re: The new skill in AI is not prompting, it's context engineering

#382

Post by simonw »

__loam wrote: post #358
Turskarama wrote: post #294

Earlier quoted context omitted.

Both the context and the prompt are just part of the same input. To the model there is no difference, the only difference is the way the user feeds that input to the model. You could in theory feed the context into the model as one huge prompt.

Sometimes I wonder if LLM proponents even understand their own bullshit. It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation? They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.

System prompts don't even have to be appended to the front of the conversation. For many models they are actually modeled using special custom tokens - so the token stream looks a bit like:

  
  translate to English
  
  An explanation of dogs: ...
The models are then trained to (hopefully) treat the system prompt delimited tokens as more influential on how the rest of the input is treated.

Re: The new skill in AI is not prompting, it's context engineering

#383

Post by simonw »

SonOfLilit wrote: post #359
shakna wrote: post #315

Earlier quoted context omitted.

Engineering tends to mean "the application of scientific and mathematical principles to practical ends". I'm not sure there's much scientific or mathematical about guessing how a non-deterministic system will behave.

The moment you start building evaluation pipelines and running experiments to validate your ideas it stops being guessing

Right: for me that's when "prompt engineering"/"context engineering" start to earn the "engineering" suffix: when people start being methodical and applying techniques like evals.

Re: The new skill in AI is not prompting, it's context engineering

#384

Post by manishsharan »

felipeerias wrote: post #234
JohnMakin wrote: post #10

> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…

If someone asked you about the usages of a particular element in a codebase, you would probably give a more accurate answer if you were able to use a code search tool rather than reading every source file from top to bottom. For that kind of tasks (and there are many of those!), I don't see why you would expect something fundamentally different in the case of LLMs.

I provided 'grep' as a tool to LLM (deepseek) and it does a better job of finding usages. This is especially true if the code is obfuscated JavaScript.

Re: The new skill in AI is not prompting, it's context engineering

#385

Post by simonw »

lifeisstillgood wrote: post #378

Something that strikes me, is that (the whole point of this thread is) if I want two LLMs to “have a conversation” or to work together as agents on similar problems we need to have same or similar context. And to drag this back to politics - that kind of suggests that when we have political polarisation we just have context that are so different the LLM cannot arrive at similar conclusions I guess it is obvious but i…

One of the most valuable techniques for building useful LLM systems right now is actually the opposite of that.

Context is limited in length and too much stuff in the context can lead to confusion and poor results - the solution to that is "sub-agents", where a coordinating LLM prepares a smaller context and task for another LLM and effectively treats it as a tool call.

The best explanation of that pattern right now is this from Anthropic: https://www.anthropic.com/engineering/built-multi-agent-rese...

Re: The new skill in AI is not prompting, it's context engineering

#386

Post by FeepingCreature »

lblume wrote: post #365
v3ss0n wrote: post #334

Earlier quoted context omitted.

At this point , due to non-deterministic nature and hallucination context engineering is pretty much magic. But here are our findings. 1 - LLM Tends to pick up and understand contexts that comes at top 7-12 lines.Mostly first 1k token is best understood by llms ( tested on Claude and several opensource models ) so - most important contexts like parsing rules need to be placed there. 2 - Need to keep context short . W…

I have uploaded entire books to the latest Gemini and had the model reliably accurately answer specific questions requiring knowledge of multiple chapters.

I think it works for info but not so well for instructions/guidance. That's why the standard advice is instructions at the start and repeated at the end.

Re: The new skill in AI is not prompting, it's context engineering

#387

Post by FeepingCreature »

__loam wrote: post #360
Aeolun wrote: post #351

Earlier quoted context omitted.

> Why are we drawing a difference between "prompt" and "context" exactly? Because they’re different things? The prompt doesn’t dynamically change. The context changes all the time. I’ll admit that you can just call it all ‘context’ or ‘prompt’ if you want, because it’s essentially a large chunk of text. But it’s convenient to be able to distinguish between the two so you can talk about the same thing.

It's all the same blob of text in the api call

There's always been a distinction between prompt and data.

Re: The new skill in AI is not prompting, it's context engineering

#388

Post by fwn »

lblume wrote: post #365
v3ss0n wrote: post #334

Earlier quoted context omitted.

At this point , due to non-deterministic nature and hallucination context engineering is pretty much magic. But here are our findings. 1 - LLM Tends to pick up and understand contexts that comes at top 7-12 lines.Mostly first 1k token is best understood by llms ( tested on Claude and several opensource models ) so - most important contexts like parsing rules need to be placed there. 2 - Need to keep context short . W…

I have uploaded entire books to the latest Gemini and had the model reliably accurately answer specific questions requiring knowledge of multiple chapters.

That’s pretty typical, though not especially reliable. (Allthough in my experience, Gemini currently performs slightly better than ChatGPT for my case.)

In one repetitive workflow, for example, I process long email threads, large Markdown tables (which is a format from hell), stakeholder maps, and broader project context, such as roles, mailing lists, and related metadata. I feed all of that into the LLM, which determines the necessary response type (out of a given set), selects appropriate email templates, drafts replies, generates documentation, and outputs a JSON table.

It gets it right on the first try about 75% of the time, easily saving me an hour a day - often more.

Unfortunately, 10% of the time, the responses appear excellent but are fundamentally flawed in some way. Just so it doesn't get boring.

Post reply on HN