Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

281–290 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#281

Post by matkoniecz »

bgwalter wrote: post #139

These discussions increasingly remind me of gamers discussing various strategies in WoW or similar. Purportedly working strategies found by trial and error and discussed in a language that is only intelligible to the in-group (because no one else is interested). We are entering a new era of gamification of programming, where the power users force their imaginary strategies on innocent people by selling them to the eq…

> only intelligible to the in-group (because no one else is interested)

that applies to basically any domain-specific terminology, from WoW raids through cancer research to computer science and say OpenStreetMap

Re: The new skill in AI is not prompting, it's context engineering

#282

Post by groestl »

ffsm8 wrote: post #263
nonethewiser wrote: post #251

Earlier quoted context omitted.

>Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "sticks" in non-deterministic space. I dont quite follow. Prompts and contexts are different things. Sure, you can get thing into contexts with prompts but that doesn't mean they are entirely the same. You could have a long running conversation with a lot in the context. A given prompt may work poorly, w…

Yeah, if anything it should be called an art. The term engineering makes little sense in this context, but really... Did it make sense for eg "QA Engineer" and all the other jobs we tacked it on, too? I don't think so, so it's kinda arguing after we've been misusing the term for well over 10 yrs

Well, to get the right thing into the context in a performant way when you dealing with a huge dataset is definitely engineering.

Re: The new skill in AI is not prompting, it's context engineering

#283

Post by ethan_smith »

mountainriver wrote: post #162

You can give most of the modern LLMs pretty darn good context and they will still fail. Our company has been deep down this path for over 2 years. The context crowd seems oddly in denial about this

We've experienced the same - even with perfectly engineered context, our LLMs still hallucinate and make logical errors that no amount of context refinement seems to fix.

Re: The new skill in AI is not prompting, it's context engineering

#284

Post by surrTurr »

Zopieux wrote: post #146
jumploops wrote: post #75

To anyone who has worked with LLMs extensively, this is obvious. Single prompts can only get you so far (surprisingly far actually, but then they fall over quickly). This is actually the reason I built my own chat client (~2 years ago), because I wanted to “fork” and “prune” the context easily; using the hosted interfaces was too opaque. In the age of (working) tool-use, this starts to resemble agents calling sub-age…

I find it hilarious that this is how the original GPT3 UI worked, if you remember, and we're now discussing of reinventing the wheel. A big textarea, you plug in your prompt, click generate, the completions are added in-line in a different color. You could edit any part, or just append, and click generate again. 90% of contemporary AI engineering these days is reinventing well understood concepts "but for LLMs", or i…

The original GPT-3 was trained very differently than modern models like GPT-4. For example, the conversational structure of an assistant and user is now built into the models, whereas earlier versions were simply text completion models.

It's surprising that many people view the current AI and large language model advancements as a significant boost in raw intelligence. Instead, it appears to be driven by clever techniques (such as "thinking") and agents built on top of a foundation of simple text completion. Notably, the core text completion component itself hasn’t seen meaningful gains in efficiency or raw intelligence recently...

Re: The new skill in AI is not prompting, it's context engineering

#286

Post by surrTurr »

Context engineering will be just another fad, like prompt engineering was. Once the context window problem is solved, nobody will be talking about it any more.

Also, for anyone working with LLMs right now, this is a pretty obvious concept and I'm surprised it's on top of HN.

Re: The new skill in AI is not prompting, it's context engineering

#287

Post by PeterStuer »

JohnMakin wrote: post #10

> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…

"these are non-deterministic machines"

Only if you choose so by allowing some degree of randomness with the temperature setting.

Re: The new skill in AI is not prompting, it's context engineering

#288

Post by Roark66 »

danans wrote: post #198
neilv wrote: post #131

> Then you can generate a response. > > Hey Jim! Tomorrow’s packed on my end, back-to-back all day. Thursday AM free if that works for you? Sent an invite, lmk if it works. Feel free to send generated AI responses like this if you are a sociopath.

Came here to say this, too - creepy. Especially when there is no person in the loop, just an LLM agent responding on someone's behalf in their voice.

Isn't the point that it prepares the response, shows it to you along with some context to you. Like a sidebar showing who the other person is with a short summary of your last comms and your calendar. It should let you move the "proposed appointment" in that sidebar calendar and it should update the response to match your choice. If it clashes and you have no time it should show you what those other things are (maybe propose what you could shift) and so on.

This is how I imagine proper AI integration.

What I also want is not sending all my data to the provider. With the model sizes we use these days it's pretty much impossible to run them locally if you want the best, so imo the company that will come up with the best way to secure customer data will win.

Re: The new skill in AI is not prompting, it's context engineering

#289

Post by therealdrag0 »

baxtr wrote: post #2

> Conclusion Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. It is about the engineering of context and providing the right information and tools, in the right format, at the right time. It’s a cross-functional challenge that involves understanding your business use case, defining your outputs, and structuring all the necessary information so that an LLM can “a…

Ya reminds me of social engineering. Like we’re seeing “How to Win Programming and Influence LLMs”.

Re: The new skill in AI is not prompting, it's context engineering

#290

Post by pbreit »

JohnMakin wrote: post #10

> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…

What's the difference?
Post reply on HN