411–420 of 550 posts
Earlier quoted context omitted.
Engineering tends to mean "the application of scientific and mathematical principles to practical ends". I'm not sure there's much scientific or mathematical about guessing how a non-deterministic system will behave.
The moment you start building evaluation pipelines and running experiments to validate your ideas it stops being guessing
> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…
It's magical thinking all the way down. Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "sticks" in non-deterministic space.
These discussions increasingly remind me of gamers discussing various strategies in WoW or similar. Purportedly working strategies found by trial and error and discussed in a language that is only intelligible to the in-group (because no one else is interested). We are entering a new era of gamification of programming, where the power users force their imaginary strategies on innocent people by selling them to the eq…
Earlier quoted context omitted.
The best way to understand a car is to build a car. Hardly anyone is going to do that, but we still all use them quite well in our daily lives. In large part because the companies who build them spend time and effort to improve them and take away friction and complexity. If you want to be an F1 driver it's probably useful to understand almost every part of a car. If you're a delivery driver, it probably isn't, even i…
yes except intelligence isn't like a car, there's no way to break the complicated emergent behaviors of these models into simple abstractions. you can understand a LLM by training one the same amount you can understand a brain by dissection.
It is probably 6-7 months ago I used ChatGPT for "vibe coding", and my main complaint was that the model eventually started moving away too far from its intended goal, as and it eventually go lost and stuck in some loop. In which case I had to fire up a new model, and feed all the context I had, and continue. A couple of days ago I fired up o4-mini-high, and I was blown away how long it can remember things, how much…
I had a data wrangling task where I determine the value of a column in a dataframe based on values in several other columns. I implemented some rules to do the matching and it worked for most of the records, but there are some data quality issues. I asked Claude Code to implement a hybrid approach with rules and ML. We discussed some features and weighting. Then, it reviewed my whole project, built the model and integrated it into what I already had. The finished process uses my rules to classify records, trains the model on those and then uses the model to classify the rest of them.
Someone had been doing this work manually before and the automated version produces a 99.3% match. AI spent a few minutes implementing this at a cost of a couple dollars and the program runs in about a minute compared to like 4 hours for the manual process it's replacing.
Earlier quoted context omitted.
“non-deterministic machines“ Not correct. They are deterministic as long as a static seed is used.
That's not true in practice. Floating point arithmetic is not commutative due to rounding errors, and the parallel operations introduce non-determinisn even at temperature 0.
Earlier quoted context omitted.
The moment you start building evaluation pipelines and running experiments to validate your ideas it stops being guessing
Right: for me that's when "prompt engineering"/"context engineering" start to earn the "engineering" suffix: when people start being methodical and applying techniques like evals.
Earlier quoted context omitted.
Only without all that pesky determinism and reproducibility. (Whoever's about to say "well ackshually temperature of zero", don't.)
You forgot about lower performance and efficiency. And longer build/run cycles. And more hardware/power usage.
(*) "like" in the sense of "not like"
> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…
At this point , due to non-deterministic nature and hallucination context engineering is pretty much magic. But here are our findings. 1 - LLM Tends to pick up and understand contexts that comes at top 7-12 lines.Mostly first 1k token is best understood by llms ( tested on Claude and several opensource models ) so - most important contexts like parsing rules need to be placed there. 2 - Need to keep context short . W…
I tested with 8B model, 14B model and 32B model.
I wanted it to create structured json, and the context was quite large like 60k tokens.
the 8B model failed miserably despite supporting 128k context, the 14b did better the 32B one almost got everything correct. However when jumping to a really large model like grok-3-mini it got it all perfect.
The 8B, 14B, 32B models I tried were Qwen 3. All the models I tested I disabled thinking.
Now for my agent workflows I use small models for most workflow (it works quite nicely) and only use larger models when the problem is harder.
Earlier quoted context omitted.
"these are non-deterministic machines" Only if you choose so by allowing some degree of randomness with the temperature setting.
They are usually nondeterministic even at temperature 0 - due to things like parallelism and floating point rounding errors.