Semantics. The context is actually part of the "prompt". Sure we can call it "context engineering" instead of "prompt engineering", where now the "prompt" is part of the "context" (instead of the "context" being part of the "prompt") but it's essentially the same thing.
The new skill in AI is not prompting, it's context engineering
431–440 of 550 posts
Re: The new skill in AI is not prompting, it's context engineering
#432Earlier quoted context omitted.
What? You can get consistent output on local models. I can train large nets deterministically too (CUBLAS flags). What your saying isn't true in practice. Hell I can also go on the anthropic API right now and get verbatim static results.
I think lots of people misunderstand that the "non-deterministic" nature of LLMs come from sampling the token distribution, not from the model itself.
It's also the way the model runs. Setting temperature to zero and picking a fixed seed would ideally result in deterministic output from the sampler, but in parallel execution of matrix arithmetic (eg using a GPU) the order of floating point operations starts to matter, so timing differences can produce different results.
Re: The new skill in AI is not prompting, it's context engineering
#433Earlier quoted context omitted.
So for safety critical systems, one should not look or check if code has been AI generated? If you don't review the code your C compiler generates now, why not? Compiler bugs still happen, you know.
You do understand that LLM output is non-deterministic and tends to have a higher error ratio than compiler bugs, which do not exhibit this “feature”. I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it.
I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it.
(Shrug) Tool use is important. People who are better than you at using tools will outcompete you. That's not an opinion or "combative," whatever that means, just the way it works.
It's no skin off my nose either way, but HN is not a place where I like to see ignorant, ill-informed opinions paraded with pride.
Re: The new skill in AI is not prompting, it's context engineering
#434LLM agents remind me of the great Nolan movie „Memento“. The agents cannot change their internal state hence they change the encompassing system. They do this by injecting information into it in such a way that the reaction that is triggered in them compensates for their immutability. For this reason I call my agents „Sammy Jenkins“.
I think we can reasonably expect they will become non-stateless in the next few years.
Re: The new skill in AI is not prompting, it's context engineering
#435Earlier quoted context omitted.
Surely claims about context engineering can also be tested using scientific methodology?
yeah, but it's a different type of science. the move from "software engineering" to "AI engineering" is basically a switch from a hard science to a soft science. rather than being chemists and physicists making very precise theory-driven predictions that are verified by experiment, we're sociologists and psychologists randomly changing variables and then doing a t-test afterward and asking "did that change anything?"
the difference is between having a "model" and a "theory". A theory tries to explain the "why" based on some givens, and a model tell you the "how". For engineering, we want why and not how. ie, for bugs, we want to root-cause, and fix - not fix by trial-and-error.
the hard sciences have theories. and soft sciences have models.
computer science is built on theory (turing machine/lambda calc/logic).
AI models are well "models" - we dont know why it works but it seems to - thats how models are.
Re: The new skill in AI is not prompting, it's context engineering
#436LLM agents remind me of the great Nolan movie „Memento“. The agents cannot change their internal state hence they change the encompassing system. They do this by injecting information into it in such a way that the reaction that is triggered in them compensates for their immutability. For this reason I call my agents „Sammy Jenkins“.
I think we can reasonably expect they will become non-stateless in the next few years.
Why?
Re: The new skill in AI is not prompting, it's context engineering
#437Earlier quoted context omitted.
I highly highly doubt that training a LLM like gpt-2 will help you use something the size of GPT-4. And I guess most people can't afford to train something like GPT-4. I trained some NNs back before the ChatGPT era, I don't think any of it helps in using Chatgpt/alternatives
With modern high-quality datasets and the plummeting H100 rental costs it is 100% a feasible undertaking for an individual to train a model with performance far closer to gpt-4-1106-preview than to gpt-2, in fact its difficult to train a model that performs as badly as gpt-2 without carefully selecting for datasets like OpenWebText with the explicit purpose of replicating runs of historical interest: modern datasets…
Ok I'm open (and happy to hear!) to being wrong on this. You are saying I can find tutorials which can train something like gpt3.5 level model (like a 7B model?) from scratch for under 1000 USD of cloud compute? Is there a guide on how to do this?
Re: The new skill in AI is not prompting, it's context engineering
#438That is prompting. It's all a prompt going in. The parts you see or don't see as an end user is just UX. Of course when you obscure things, it changes the UX for the better or the worse.
Re: The new skill in AI is not prompting, it's context engineering
#439Only more mental exercises to avoid reading the writing on the wall: LLM DO NOT REASON ! THEY ARE TOKEN PREDICTION MACHINES Thank you for your attention in this matter!
What is reasoning? And how is it apparent that LLMs can't reason? The reality for me is that they are not perfect at reasoning and have many quirks, but it seems to be that they are able to form new conclusions based on provided premises. Genuinely curious why you think they can't.
[deleted]
Re: The new skill in AI is not prompting, it's context engineering
#440Earlier quoted context omitted.
> Not really. Got some code you don't understand? Feed it to a model and ask it to add comments. Absolutely not. An experienced individual in their field can tell if the AI made a mistake in the comments / code rather than the typical untrained eye. So no, actually read the code and understand what it does. > Ultimately humans will never need to look at most AI-generated code, any more than we have to look at the mac…
So for safety critical systems, one should not look or check if code has been AI generated? If you don't review the code your C compiler generates now, why not? Compiler bugs still happen, you know.
> If you don't review the code your C compiler generates now, why not?
That isn't a reason why you should NOT review AI-generated code. Even when comparing the two, a C compiler is far more deterministic in the code that it generates than LLMs, which are non-deterministic and unpredictable by design.
> Compiler bugs still happen, you know.
The whole point is 'verification' which is extremely important in compiler design and there exists a class of formally-verified compilers that are proven to not generate compiler bugs. There is no equivalent for LLMs.
In any case, you still NEED to check if the code's functionality matches the business requirements; AI-generated or not; especially in safety critical systems. Otherwise, it is considered as a logic bug in your implementation.