> I've literally never heard anyone conflate an agent and a model. Ever.Author is a senior staff engineer. A big part of his job is to help his org's engineers and non-technical decision makers understand basics. My assumption (because it's my experience) is that he is writing this piece because has seen this confusion in others > N times.
And to be fair to those juniors and less-technical folks: big groups of AI phds make the same mistake.
Papers in prominent ML conferences up to like 2024-2025 that were consistently presenting comparisons of proprietary model end-points to open weight models as if they were apples-apples comparisons. Even though there were obvious examples of prop model providers doing "stuff" behind the endpoint as early as Fall 2021.
Most of those papers did not put in the legwork to figure out if/when that "stuff" was happening, or work to figure out what the "stuff" probably was, or conjecture to what extent that "stuff" might overlap with the benchmarks being used, or try to add basic interventions to open-weight models to get a better comparison point, or even simply state the caveat that they don't actually know what happens when they hit a chat completions endpoint!
Not exactly the same thing as conflating agent with model, but 100% conflating "model+software" with "model", which is at the end of the day basically the same confusion. And those papers were being written by and for llm nerds. So if ICML/NeurIPS authors and reviewers aren't careful about "software+model" vs "model" in stating and evaluating their claims, it's unsurprising that some a SWE or non-CS SME also doesn't understand how the LLM software stack works.
It doesn't help that some model providers (OpenAI...) actively obfuscate how the stack works in their marketing.