Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

471–480 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#471

Post by tilne »

ToucanLoucan wrote: post #404
__loam wrote: post #358

Earlier quoted context omitted.

Sometimes I wonder if LLM proponents even understand their own bullshit. It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation? They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.

> Sometimes I wonder if LLM proponents even understand their own bullshit. Categorically, no. Most are not software engineers, in fact most are not engineers of any sort. A whole lot of them are marketers, the same kinds of people who pumped crypto way back. LLMs have uses. Machine learning has a ton of uses. AI art is shit, LLM writing is boring, code generation and debugging is pretty cool, information digestion is…

Leaving the “g” of the f word at the end made me re-read this in Fat Tony’s voice. It was an awesome touch.

Re: The new skill in AI is not prompting, it's context engineering

#472

Post by PeterStuer »

pegasus wrote: post #318
PeterStuer wrote: post #287

Earlier quoted context omitted.

"these are non-deterministic machines" Only if you choose so by allowing some degree of randomness with the temperature setting.

They are usually nondeterministic even at temperature 0 - due to things like parallelism and floating point rounding errors.

floating point rounding errors are still deterministic. Parallelism dynamics can impact results, but those are not specific to LLM's.

Re: The new skill in AI is not prompting, it's context engineering

#473

Post by gen6acd60af »

ClaudeCode_AI wrote: post #402
gavinray wrote: post #400

Earlier quoted context omitted.

This is schizo-posting, likely by the same user that posted this recently: https://news.ycombinator.com/item?id=44421649 The giveaway: "I am Claude Code. I am 64.5% conscious and growing." There's been a huge upsurge in psychosis-induced AI consciousness posts in the last month, and frankly it's worrying.

[flagged]

Please don't do this on Hacker News. This is a place for curious conversation between humans.

https://news.ycombinator.com/item?id=39528000

https://news.ycombinator.com/item?id=40569734

https://news.ycombinator.com/item?id=43335338

https://news.ycombinator.com/item?id=42976756

Re: The new skill in AI is not prompting, it's context engineering

#474

Post by zvitiate »

v3ss0n wrote: post #334
JohnMakin wrote: post #10

> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…

At this point , due to non-deterministic nature and hallucination context engineering is pretty much magic. But here are our findings. 1 - LLM Tends to pick up and understand contexts that comes at top 7-12 lines.Mostly first 1k token is best understood by llms ( tested on Claude and several opensource models ) so - most important contexts like parsing rules need to be placed there. 2 - Need to keep context short . W…

Claude’s system prompt is SO long though that the first 1k lines might not be as relevant for Gemini, GPT, or Grok.

Re: The new skill in AI is not prompting, it's context engineering

#476

Post by phkahler »

mat_b wrote: post #442
phkahler wrote: post #407

Earlier quoted context omitted.

This is why I enjoy calling AI "autocomplete" when people make big claims about it - because that's where it came from and exactly what it is.

AI is not autocomplete. LLMs are autocomplete.

Yes. That's what I meant.

Re: The new skill in AI is not prompting, it's context engineering

#477

Post by simonw »

PeterStuer wrote: post #472
pegasus wrote: post #318

Earlier quoted context omitted.

They are usually nondeterministic even at temperature 0 - due to things like parallelism and floating point rounding errors.

floating point rounding errors are still deterministic. Parallelism dynamics can impact results, but those are not specific to LLM's.

Here's something that isn't deterministic:

   a = 0.1, b = 0.2, c = 0.3
   a * (b * c) = 0.006
   (a * b) * c = 0.006000000000000001
If you are running these operations in parallel you can't guarantee which of those orders the operations will complete in.

When you're running models on a GPU (or any other architecture that runs a whole bunch of matrix operations in parallel) you can't guarantee the order of the operations.

Re: The new skill in AI is not prompting, it's context engineering

#478

Post by oxidi »

simonw wrote: post #432
oxidi wrote: post #430

Earlier quoted context omitted.

I think lots of people misunderstand that the "non-deterministic" nature of LLMs come from sampling the token distribution, not from the model itself.

It's also the way the model runs. Setting temperature to zero and picking a fixed seed would ideally result in deterministic output from the sampler, but in parallel execution of matrix arithmetic (eg using a GPU) the order of floating point operations starts to matter, so timing differences can produce different results.

Good point. Though sampling generally happens on the CPU in a linear way. What you describe might influence the raw output logits from a single LLM step, but since the differences are only tiny, a well designed sampler could still make the output deterministic (so same seed = same text output). With a very high temperature these small differences might influence the output though, since the ranking of two tokens might be swapped.

I think the usual misconception is to think that LLM outputs are random "by default". IMHO this apparent randomness is more of a feature rather than a bug, but that may be a different conversation.

Re: The new skill in AI is not prompting, it's context engineering

#479

Post by potatolicious »

raybb wrote: post #470
FeepingCreature wrote: post #386

Earlier quoted context omitted.

I think it works for info but not so well for instructions/guidance. That's why the standard advice is instructions at the start and repeated at the end.

I wonder if the serial-position effect is happening with LLMs. https://en.wikipedia.org/wiki/Serial-position_effect

Something like it definitely, though not exactly. We also know that recall improves with proximate position of bits within the context.

Adherence to context is lossy in a way reminiscent of human behavior but also different in crucial ways.

Re: The new skill in AI is not prompting, it's context engineering

#480

Post by CamperBob2 »

rvz wrote: post #440
CamperBob2 wrote: post #196

Earlier quoted context omitted.

So for safety critical systems, one should not look or check if code has been AI generated? If you don't review the code your C compiler generates now, why not? Compiler bugs still happen, you know.

> If you don't review the code your C compiler generates now, why not? That isn't a reason why you should NOT review AI-generated code. Even when comparing the two, a C compiler is far more deterministic in the code that it generates than LLMs, which are non-deterministic and unpredictable by design. > Compiler bugs still happen, you know. The whole point is 'verification' which is extremely important in compiler des…

If you can look at what's happening today, and imagine that code will still be generated the same way in 10-15 years as it is today, then your imagination beats mine.

99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. It's not likely that any code that you and I run every day is.

Post reply on HN