Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

511–520 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#511

Post by CamperBob2 »

rvz wrote: post #499
CamperBob2 wrote: post #480

Earlier quoted context omitted.

If you can look at what's happening today, and imagine that code will still be generated the same way in 10-15 years as it is today, then your imagination beats mine. 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. It's not likely that any code that you and I run every day is.

> 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. Again, that isn't a reason to never check or write tests for your code because an "AI-generated it" or even assuming that an AI will detect all of them. In fact, it means you NEED to do more reviewing, checking and testing than ever before. > It's not likely that any code that you and I run every day is. S…

Yes, I'm very sure. 99.9999% of the code you are running is not formally proven to be correct, and was not generated by a compiler whose output was formally proven to be correct.

Just curious, how much time have you spent in (a) industry, (b) a CS classroom, or (c) both?

Re: The new skill in AI is not prompting, it's context engineering

#512

Post by daxfohl »

daxfohl wrote: post #503
daxfohl wrote: post #501

Earlier quoted context omitted.

I'm surprised there isn't already an ecosystem of libraries that just do this. When building agents you either have to roll your own or copy an algorithm out of some article. I'd expect this to be a lot more plug and play, and as swappable as LLMs themselves by EOY, along with a bunch of tooling to help with observability, A/B testing, cost and latency analysis (since changing context kills the LLM cache), etc.

Or maybe it's that each of these things is pretty simple in itself. Clipping context is one line of code, summarizing could be a couple lines to have an LLM summarize it for you, etc. So not substantial enough for a formal library. Whereas the combinations of these techniques is very application dependent, so not reusable enough to warrant separating as an independent library. Or maybe it just hasn't matured yet and…

Though in a way, this feels similar to things like garbage collection, disk defragmentation, or even query planning. Yes, you could build libraries that do these sorts of things for you, but in all likelihood the LLM providers will embed custom-built versions of them that have been battle tested and trained thoroughly to interop well with the corresponding LLM. So whole there could still be an ecosystem, it would likely be a fairly niche thing for very specific use cases or home-grown LLMs.

Maybe something like the equivalent of AWS Firecracker for whatever the equivalent of AWS Lambda is in the future LLM world.

Re: The new skill in AI is not prompting, it's context engineering

#514

Post by yahoozoo »

lblume wrote: post #365
v3ss0n wrote: post #334

Earlier quoted context omitted.

At this point , due to non-deterministic nature and hallucination context engineering is pretty much magic. But here are our findings. 1 - LLM Tends to pick up and understand contexts that comes at top 7-12 lines.Mostly first 1k token is best understood by llms ( tested on Claude and several opensource models ) so - most important contexts like parsing rules need to be placed there. 2 - Need to keep context short . W…

I have uploaded entire books to the latest Gemini and had the model reliably accurately answer specific questions requiring knowledge of multiple chapters.

Mind if I ask how you’re doing this? I have uploaded short stories of <40,000 words in .txt format and when I ask questions like “How many chapters are there?” or “What is the last sentence in the story?” it gets it wrong. If I paste a chapter or two at a time then ask, it works better, but that’s tedious…

Re: The new skill in AI is not prompting, it's context engineering

#515

Post by welshwelsh »

jedimastert wrote: post #369
ozgung wrote: post #319

Earlier quoted context omitted.

Why not? What is so special about reasoning that you cannot achieve by predicting tokens aka. constructing sentences?

Predicting tokens and constructing sentences are not the same thing. It cannot create its own sentences because it does not have a self

Humans also do not have a self, merely the illusion of a self.

Re: The new skill in AI is not prompting, it's context engineering

#516

Post by [deleted] »

yahoozoo wrote: post #514
lblume wrote: post #365

Earlier quoted context omitted.

I have uploaded entire books to the latest Gemini and had the model reliably accurately answer specific questions requiring knowledge of multiple chapters.

Mind if I ask how you’re doing this? I have uploaded short stories of <40,000 words in .txt format and when I ask questions like “How many chapters are there?” or “What is the last sentence in the story?” it gets it wrong. If I paste a chapter or two at a time then ask, it works better, but that’s tedious…

[deleted]

Re: The new skill in AI is not prompting, it's context engineering

#517

Post by Davidzheng »

benreesman wrote: post #505
Davidzheng wrote: post #491

Earlier quoted context omitted.

I'm in agreement--RLHF won't lead to massively more intelligent beings than humans. But I said RL not RLHF

Well what you said is: "On the contrary, I believe in every verifiable domain RL must drive the agent to be the most intelligent (relative to RL award) it can be under the constraints--and often it must become more intelligent than humans in that environment." And I said it's not that simple, in no way demonstrated, unlikely with current technology, and basically, nope.

Ah you're worried about convergence issues? My (Bad) understanding was that the self-driving car stuff is more about inadequacies of models in which you simulate training and data collection than convergence of algorithms but I could be wrong. I mean that statement was just a statement that I think you can get RL to converge to close to optimum--which I agree is a bit of a stretch as RL is famously finicky. But I don't see why one shouldn't expect this to happen as we tune the algorithms.

Re: The new skill in AI is not prompting, it's context engineering

#518

Post by skydhash »

gpm wrote: post #409
skydhash wrote: post #354

Earlier quoted context omitted.

But why not provide the search tool instead of being an imperfect interface between it and the person asking? The only reason for the latter is that you have more applied knowledge in the context and can use the tool better. For any other case, the answer should be “use this tool”.

Because the LLM is faster at typing the input, and faster at reading the output, than I am... the amount of input I have to give the LLM is less than what I have to give the search tool invocations, and the amount of output I have to read from the LLM is less than the amount of output from the search tool invocations. To be fair it's also more likely to mess up than I am, but for reading search results to get an idea…

So instead of building better tool, we're patching the last one with another tool that is not even reliable, just using it faster.

That reminds me of the first chapter in "The Programmer Brain" by Felienne Hermans. There's an explanation there that confusion when reading code is caused by three things:

- Lack of knowledge: When you don't have the faintest idea of the notation or symbol being used, aka the WHAT.

- Lack of information: When you know the WHAT, but you can't figure out the WHY.

- Lack of processing power: When you have an idea of the WHY, but can't grasp the HOW.

We already have methods and tooling for all the above and they work fine without having to do shamanic rituals.

Re: The new skill in AI is not prompting, it's context engineering

#519

Post by skydhash »

Daub wrote: post #510
skydhash wrote: post #241

Earlier quoted context omitted.

> artists and designers lack the consistent terminology to describe what they are doing I don't think they do. It may not be completely consistent, but open any art book and you find the same thing being explained again and again. Just for drawing humans, you will find emphasis on the skeleton and muscle volume for forms and poses, planes (especially the head) for values and shadows, some abstract things like stabili…

I concur that there is, on some matters, a general agreement in art books. However, certainly it does not help that there is so much inconsistency of terminology. For example: the way that hue and color are so frequently used interchangeably, likewise lightness, brightness, tone and value. What bothers me more is that so much truly important material is not being addressed as explicitly as it should be. For example:…

> For example, a common device that painters employ is to configure the neighboring regional contrast of a form can be light against dark on one edge and dark against light on the opposing edge.

I'm not fully sure of what you means. If we take the following example, are you talking about the neck and the collar of the girl?

https://i.pinimg.com/originals/ea/70/0b/ea700b6a0b366c13187e...

https://fr.pinterest.com/pin/453596993695189968/

I think the name of the concept is "edge control" (not really original). You can find some explanation here

https://www.youtube.com/watch?v=zpSlGmbUB08

To keep it short, there's no line in reality. So while you can use them when sketching, they are pretty crude, kinda like a piano with only 2 keys. The best thing is edges, meaning the delimitation between two contrasting area. If you're doing grayscale, your areas are values (light and shadow) and it's pretty easy. Once you add color, there's more dimension to play with and it became very difficult (warm and cold color, atmospheric colors, brush stroke that gives the illusion of details,...).

Again, this falls under the things that are easy to explain, but take a while to be able to observe it and longer to reproduce it.

There's a book called "Color and Light" by James Gurney that goes in depth about all of these. There's a lot of parameters that goes inside a brush stroke in a specific area of a painting.

Re: The new skill in AI is not prompting, it's context engineering

#520

Post by hackable_sand »

pyman wrote: post #167
benreesman wrote: post #154

The new skill is programming, same as the old skill. To the extent these things are comprehensible, you understand them by writing programs: programs that train them, programs that run inferenve, programs that analyze their behavior. You get the most out of LLMs by knowing how they work in detail. I had one view of what these things were and how they work, and a bunch of outcomes attached to that. And then I spent a…

Saying the best way to understand LLMs is by building one is like saying the best way to understand compilers is by writing one. Technically true, but most people aren't interested in going that deep.

It's not that deep
Post reply on HN