11–20 of 266 posts
> It helps to know that LLMs don’t “reason”. They predict .. Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.
> It helps to know that LLMs don’t “reason”. They predict .. Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.
> In the past year, agent harnesses crossed the “can it be done” rubicon. Brother, I'm still in "Can you get it right?"-mode. What am I doing wrong? (Rhetorical, but advice welcomed).
Tell it to use red/green TDD and start things off with an already configured test suite, maybe with a single test that asserts 1+1==2. Make sure it know how to run the tests before it starts writing any additional code. Then set it a clear goal.
> In the past year, agent harnesses crossed the “can it be done” rubicon. Brother, I'm still in "Can you get it right?"-mode. What am I doing wrong? (Rhetorical, but advice welcomed).
I’ve found some success is small projects, with limited scope, in a greenfield. I’m terrified to attempt agentic anything in the repo my job actually cares about. I triggered it once by accident, when the agent was first rolled out and enabled by default… it broke everything. Now I just use ask mode, and even that is wrong half the time, and once it goes wrong it just keeps getting worse. I saw a post from Dave Plumb…
This sounds like programming but with extra steps that make it take longer with less reliability.
With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt. Subtle things like which error states are "oh shit we need to bail" vs "this isn't a deal breaker." Sometimes it will ask, but more often than not it…
The error types and codes, it will produce to spec.
If you type 'make me that thingy' - yes, it's probably not going to do what you want, but if you give it spec and guidance, it usually will.
The 'interface design' ... not very good though.
Earlier quoted context omitted.
Tell it to use red/green TDD and start things off with an already configured test suite, maybe with a single test that asserts 1+1==2. Make sure it know how to run the tests before it starts writing any additional code. Then set it a clear goal.
Basically all the examples of LLM's building impressive things have been because they have human written tests to base the implementation on. If you have an LLM write the tests the results are far less impressive or valuable.
> It helps to know that LLMs don’t “reason”. They predict .. Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.
Why would "reasoning" be an emergent property of prediction?
> It helps to know that LLMs don’t “reason”. They predict .. Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.
Why would "reasoning" be an emergent property of prediction?
> It helps to know that LLMs don’t “reason”. They predict .. Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.
Why would "reasoning" be an emergent property of prediction?
> In the past year, agent harnesses crossed the “can it be done” rubicon. Brother, I'm still in "Can you get it right?"-mode. What am I doing wrong? (Rhetorical, but advice welcomed).