Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

501–510 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#501

Post by daxfohl »

simonw wrote: post #3

I wrote a bit about this the other day: https://simonwillison.net/2025/Jun/27/context-engineering/ Drew Breunig has been doing some fantastic writing on this subject - coincidentally at the same time as the "context engineering" buzzword appeared but actually unrelated to that meme. How Long Contexts Fail - https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-ho... - talks about the various ways in which longer…

I'm surprised there isn't already an ecosystem of libraries that just do this. When building agents you either have to roll your own or copy an algorithm out of some article.

I'd expect this to be a lot more plug and play, and as swappable as LLMs themselves by EOY, along with a bunch of tooling to help with observability, A/B testing, cost and latency analysis (since changing context kills the LLM cache), etc.

Re: The new skill in AI is not prompting, it's context engineering

#502

Post by LtWorf »

coderatlarge wrote: post #467
LtWorf wrote: post #410

Earlier quoted context omitted.

Our new CTO decided to move to agile and scrum, in an effort to reduce efficiency and morale. He doesn't even take responsibility for it, but claims the board told him to do that.

is it supposedly to “improve velocity “?

Is it supposed to add 30% overhead?

Re: The new skill in AI is not prompting, it's context engineering

#503

Post by daxfohl »

daxfohl wrote: post #501
simonw wrote: post #3

I wrote a bit about this the other day: https://simonwillison.net/2025/Jun/27/context-engineering/ Drew Breunig has been doing some fantastic writing on this subject - coincidentally at the same time as the "context engineering" buzzword appeared but actually unrelated to that meme. How Long Contexts Fail - https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-ho... - talks about the various ways in which longer…

I'm surprised there isn't already an ecosystem of libraries that just do this. When building agents you either have to roll your own or copy an algorithm out of some article. I'd expect this to be a lot more plug and play, and as swappable as LLMs themselves by EOY, along with a bunch of tooling to help with observability, A/B testing, cost and latency analysis (since changing context kills the LLM cache), etc.

Or maybe it's that each of these things is pretty simple in itself. Clipping context is one line of code, summarizing could be a couple lines to have an LLM summarize it for you, etc. So not substantial enough for a formal library. Whereas the combinations of these techniques is very application dependent, so not reusable enough to warrant separating as an independent library.

Or maybe it just hasn't matured yet and we'll see more of it in the future. We'll see.

Re: The new skill in AI is not prompting, it's context engineering

#504

Post by benreesman »

Davidzheng wrote: post #492
benreesman wrote: post #446

Earlier quoted context omitted.

The literally watch it on a live stream version does in fact start with the GPT-2 arch (but evals way better): https://youtu.be/l8pRSuU81PU Lambda Labs full metas jacket accelerated interconnect clusters: https://lambda.ai/blog/introducing-lambda-1-click-clusters-a ... FineWeb-2 has versions with Llama-range token counts: https://huggingface.co/datasets/HuggingFaceFW/fineweb-2 Ray Train is one popular choice for goin…

Thanks for the links! Hopefully this doesn't come across as confrontational (this is really something I would like to try myself) but I don't think a gpt2 arch will get to close to gpt3.5 level intelligence? I feel like there was some boundary around gpt3.5 where the stuff started to feel slightly magical for me [maybe it was only the RLHF effect]. Do you think models in gpt2 size now are getting to that capability?…

I think you'll be surprised if you see the lift karpathy demonstrates from `fineweb.edu` vs `webtext` (he went back later and changed the `nanogpt` repository to use `openwebtext` because it was different enough that it wasn't a good replication of GPT-2).

But from an architecture point of view, you might be surprised at how little has changed. Rotary and/or alibi embeddings are useful, and there's a ton on the inference efficiency side (GQA -> MHA -> MLA), but you can fundamentally take a llama and start it tractably small, and then make it bigger.

You can also get checkpoint weights for tons of models that are trivially competitive, and tune heads on them for a fraction of the cost.

This leaked Google memo is a pretty good summary (and remarkably prescient in terms of how it's played out): https://semianalysis.com/2023/05/04/google-we-have-no-moat-a...

I hope I didn't inadvertently say or imply that you can make GPT-4 in a weekend, that's not true. But you can make models with highly comparable characteristics based on open software, weights, training sets, and other resources that are basically all on HuggingFace: you can know how it works.

GPT-2 is the one you can do completely by yourself starting from knowing a little Python in one day.

Re: The new skill in AI is not prompting, it's context engineering

#505

Post by benreesman »

Davidzheng wrote: post #491
benreesman wrote: post #466

Earlier quoted context omitted.

Eh...kinda. The RL in RLHF is a very different animal than the RL in a Waymo car training pipeline, which is sort of obvious when you see that the former can be done by anyone with some clusters and some talent, and the latter is so hard that even Waymo has a marked preference for operating in July in Chandler AZ: everyone else is in the process of explaining why they didn't really want Level 5 per se anyways: all br…

I'm in agreement--RLHF won't lead to massively more intelligent beings than humans. But I said RL not RLHF

Well what you said is:

"On the contrary, I believe in every verifiable domain RL must drive the agent to be the most intelligent (relative to RL award) it can be under the constraints--and often it must become more intelligent than humans in that environment."

And I said it's not that simple, in no way demonstrated, unlikely with current technology, and basically, nope.

Re: The new skill in AI is not prompting, it's context engineering

#506

Post by jeremyjh »

kachapopopow wrote: post #345

I'll quote myself since it seems oddly familiar: --- Forget AI "code", every single request will be processed BY AI! People aren't thinking far enough, why bother with programming at all when an AI can just do it? It's very narrow to think that we will even need these 'programmed' applications in the future. Who needs operating systems and all that when all of it can just be AI. In the future we don't even need hardw…

How does the AI open and close circuits without machine code?

Answer: Its AI all the way down.

Re: The new skill in AI is not prompting, it's context engineering

#507

Post by Dylan16807 »

simonw wrote: post #143
refulgentis wrote: post #141

Earlier quoted context omitted.

I agree with you, but would echo OP's concern, in a way that makes me feel like a party pooper, but, is open about what I see us all expressing squeamish-ness about. It is somewhat bothersome to have another buzz phrase. I don't why we are doing this, other than there was a Xeet from the Shopify CEO, QT'd approvingly by Karpathy, then its written up at length, and tied to another set of blog posts. To wit, it went fr…

The way I see it we're trying to rebrand because the term "prompt engineering" got redefined to mean "typing prompts full of stupid hacks about things like tipping and dead grandmas into a chatbot".

If that's what it takes to make good results, then it's respectable work even if the details are stupid.

Re: The new skill in AI is not prompting, it's context engineering

#508

Post by Terr_ »

StevenWaterman wrote: post #380
__loam wrote: post #358

Earlier quoted context omitted.

Sometimes I wonder if LLM proponents even understand their own bullshit. It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation? They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.

Yep, every AI call is essentially just asking it to predict what the next word is after: You are a helpful assistant. Why is the sky blue? Because of Rayleigh scattering. The blue light refracts more. Why is it red at sunset then? And we keep repeating that until the next word is ` `, then extract the bit in between the last assistant tags, and return it. The AI has been trained to look at ` ` differently to ` `, but…

Yeah, ultimately it's Make Document Longer machine, and in many cases it's a hidden mad-libs script behind the scenes, where your question becomes "Next the User said", and some regular code is looking for "Next the Computer said" and "performing" it at you.

In other words, there's a deliberate illusion going on where we are encouraged to believe that generating a document about a character is the same as that character being a real entity.

Re: The new skill in AI is not prompting, it's context engineering

#509

Post by bsoles »

> "the art of providing all the context for the task to be plausibly solvable by the LLM.”

And who is going to do that? The "context engineer", who doesn't know anything about the subject and runs to the LLM for quick answers without having any ability to evaluate if the answer is solid or not?

We saw the same story with "data scientists". A general understanding of tools with no understanding of the specific application areas is bound to result in crappy products, if not in business disasters.

Re: The new skill in AI is not prompting, it's context engineering

#510

Post by Daub »

skydhash wrote: post #241
Daub wrote: post #230

Earlier quoted context omitted.

For visual art I feel that the existing approaches in context engineering are very much lacking. An Ai understands well enough such simple things as content (bird, dog, owl etc), color (blue green etc) and has a fair understanding of foreground/background. However, the really important stuff is not addressed. For example: in form, things like negative shape and overlap. In color contrast things like Ratio contrast an…

> artists and designers lack the consistent terminology to describe what they are doing I don't think they do. It may not be completely consistent, but open any art book and you find the same thing being explained again and again. Just for drawing humans, you will find emphasis on the skeleton and muscle volume for forms and poses, planes (especially the head) for values and shadows, some abstract things like stabili…

I concur that there is, on some matters, a general agreement in art books. However, certainly it does not help that there is so much inconsistency of terminology. For example: the way that hue and color are so frequently used interchangeably, likewise lightness, brightness, tone and value.

What bothers me more is that so much truly important material is not being addressed as explicitly as it should be. For example: the exaggeration of contrast on which so much art relies exists in two dimensions: increase of difference and decrease of difference.

This application of contrast/affinity is a general principle that runs through the entirety of art. Indeed, I demonstrate it to my students by showing its application in Korean TV dramas. The only explicit mention I can find of this in art literature is in the work of Ruskin, nearly 200 years ago!

Even worse is that so much very important material is not being addressed at all. For example, a common device that painters employ is to configure the neighboring regional contrast of a form can be light against dark on one edge and dark against light on the opposing edge. In figurative paintings and in classic portrait photography this device is almost ubiquitous, yet as far as I am able to determine no one has named it or even written about it. We were obliged to name it ourselves (tone wrap).

> They are not difficult to explain, they are just difficult to master.

Completely agree that they can be difficult to master. However, a thing cannot be satisfactorily explained unless there is consistent (or even existent) terminology for that thing.

> So unless you can solve judgement (which styles derive from)

Nicely put.

Post reply on HN