>"What’s the state of the art of reverse engineering source code from binaries in the age of agentic coding?"
This is the original comment I was responding to, we were talking about state of the art Agentic models. So not generalizing to other scenarios.
>Sure you and they aren't skeptics and acknowledge this will get better
I think this is a common bipartisan trap where you lose a lot of nuance. And it's imprecise, you don't know whether I'm a skeptic or not. It's like reading a nuanced opinion and trying to see if they are Republican so you can agree or Democrats so you can disagree.
>Plus to blame the LLM when they haven't optimized the system prompt is IMHO quite silly - it's kind of like "did you read the instructions you were giving it?"
I think the context here is that when using agentic tools like Claude Code, you don't control the system prompt. You could write your own prompts and use naked API calls, but that's always more expensive, (because it's subsidized), and I'm not sure what the quality of that is.
The bottom line is that API calls, where you can fully control the system prompt, are more expensive. And using your OpenAI/Anthropic subscription has a fixed cost. So in that context they don't control the system prompt.
Even in cases where you could control the system prompt and use the API, there's the fact that some models (the state of the art) are fine tuned for specific tool use, so they have a bias towards fine tuned tool use. The claim is not that they are "incapable of doing X thing" it's that it's a bias towards the usage that was known at train-time or fine-tune time, instead of at inference which is much weaker. Nuance.
>Instead of blaming the model intrinsically.
Again, not blaming or being a skeptic here, just analyzing the state of the art and its current weakness, it's likely that these things are going to be improved in the next generation, this is going to move fast, if you conflate any criticism of the tools with "skepticism" you are going to miss the nuance.
>In contrast, I've seen coding agents figure out extremely complex systems problems that are clearly outside of their training set - by using tools and interacting with complex environments, and reasoning and figuring it out.
Yeah for sure, I'll give you a concrete example on this point where we agree. I made a model download a webdriver for a browser and taught it to use the webdriver to open the site, take screenshots, and evaluate how it looks visually, in addition to actually clicking buttons and navigating it. This is a great improvement when the traditional approach is just to generate frontend code and trust that it works (which to be fair, sometimes works great, but you know, it's better if it see that.)
And it works, until it doesn't and I have to remind it that it can do that. It's just a bias. If they would have trained the model with WebDriver tool access, the model would use it much more (and perhaps they are already doing that and we will see it in the next model.)
The main thesis is that instructions taught at train time 'work better' than at fine-tune time which in turn are stronger than 'inference'. To be very specific during inference tool use is much more likely immediately after mentioning it, it might be stronger more consistently at the system prompt, (but it competes with other system prompt instructions and it's still inference based). To say nothing of the costs associated with adding to inference tokens, compared to essentially free training/finetune biases. I don't think anyone disagrees that stuff you teach the model during training has better quality and less cost than stuff you teach at inference.
I think playing around with logit biases is an underrated tool to increase and control frequency of certain tools, but it doesn't seem that's being used much in this generation of vibecode tools, the interface is almost entirely textual (with some /commands starting to surface). Maybe the next generation will have the option to configure some specific parameters instead of entirely relying on textual prompting.