Let's grant that context engineering is here to stay and that we can never have context lengths be large enough to throw everything in it indiscriminately. Why is this not a perfect palce to train another AI whose job is to provide the context for the main AI?
The new skill in AI is not prompting, it's context engineering
371–380 of 550 posts
Re: The new skill in AI is not prompting, it's context engineering
#372Earlier quoted context omitted.
prediction is the result of reasoning
No it's not. Prediction is the ability to predict something. Reasoning is the ability to reason.
That's a circular definition. Can you define "reason" or "reasoning" without using the other term?
I think your definition of "reasoning" may be "think like a human" - in which case obviously LLMs can't reason because they aren't human.
Re: The new skill in AI is not prompting, it's context engineering
#373Just yesterday I was thinking if we need a code comment system that separates intentional comments from ai note/thoughts comments when working in the same files.
I don't want to delete all thoughts right away as it makes it easier for the AI to continue but I also don't want to weed trhough endless superfluous comments
Re: The new skill in AI is not prompting, it's context engineering
#374The new skill is programming, same as the old skill. To the extent these things are comprehensible, you understand them by writing programs: programs that train them, programs that run inferenve, programs that analyze their behavior. You get the most out of LLMs by knowing how they work in detail. I had one view of what these things were and how they work, and a bunch of outcomes attached to that. And then I spent a…
I highly highly doubt that training a LLM like gpt-2 will help you use something the size of GPT-4. And I guess most people can't afford to train something like GPT-4. I trained some NNs back before the ChatGPT era, I don't think any of it helps in using Chatgpt/alternatives
Re: The new skill in AI is not prompting, it's context engineering
#375After a recent conversation here, I spent a few weeks using agents. These agents are just as disappointing as what we had before. Except now I waste more time getting bad results, though I’m really impressed by how these agents manage to fuck things up. My new way of using them is to just go back to writing all the code myself. It’s less of a headache.
Which definition of "agents" are you using there, and which ones did you try?
Re: The new skill in AI is not prompting, it's context engineering
#376Earlier quoted context omitted.
Saying the best way to understand LLMs is by building one is like saying the best way to understand compilers is by writing one. Technically true, but most people aren't interested in going that deep.
The best way to understand a car is to build a car. Hardly anyone is going to do that, but we still all use them quite well in our daily lives. In large part because the companies who build them spend time and effort to improve them and take away friction and complexity. If you want to be an F1 driver it's probably useful to understand almost every part of a car. If you're a delivery driver, it probably isn't, even i…
yes except intelligence isn't like a car, there's no way to break the complicated emergent behaviors of these models into simple abstractions. you can understand a LLM by training one the same amount you can understand a brain by dissection.
Re: The new skill in AI is not prompting, it's context engineering
#377> Conclusion Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. It is about the engineering of context and providing the right information and tools, in the right format, at the right time. It’s a cross-functional challenge that involves understanding your business use case, defining your outputs, and structuring all the necessary information so that an LLM can “a…
I think too much context is harmful
Re: The new skill in AI is not prompting, it's context engineering
#378Something that strikes me, is that (the whole point of this thread is) if I want two LLMs to “have a conversation” or to work together as agents on similar problems we need to have same or similar context.
And to drag this back to politics - that kind of suggests that when we have political polarisation we just have context that are so different the LLM cannot arrive at similar conclusions
I guess it is obvious but it is also interesting
Re: The new skill in AI is not prompting, it's context engineering
#379Earlier quoted context omitted.
That's not true in practice. Floating point arithmetic is not commutative due to rounding errors, and the parallel operations introduce non-determinisn even at temperature 0.
What? You can get consistent output on local models. I can train large nets deterministically too (CUBLAS flags). What your saying isn't true in practice. Hell I can also go on the anthropic API right now and get verbatim static results.
"Hell I can also go on the anthropic API right now and get verbatim static results."
How?
Setting temperature to 0 won't guarantee the exact same output for the exact same input, because - as the previous commenter said - floating point arithmetic is non-commutative, which becomes important when you are running parallel operations on GPUs.
Re: The new skill in AI is not prompting, it's context engineering
#380Earlier quoted context omitted.
Both the context and the prompt are just part of the same input. To the model there is no difference, the only difference is the way the user feeds that input to the model. You could in theory feed the context into the model as one huge prompt.
Sometimes I wonder if LLM proponents even understand their own bullshit. It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation? They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.
Yep, every AI call is essentially just asking it to predict what the next word is after:
You are a helpful assistant.
Why is the sky blue?
Because of Rayleigh scattering. The blue light refracts more.
Why is it red at sunset then?
It's all prompt, it can all be engineered. Hell, you can even get a long way by pre-filling the start of the Assistant response. Usually works better than a system message. That's prompt engineering too.