Anthropic keeps advertising its MCP (Model Context Protocol), but to the extent it doesn't support other LLMs, e.g. GPT, it couldn't possibly gain adoption. I have yet to see any example of MCP that can be extended to use a random LLM.
Building Effective "Agents"
21–30 of 130 posts
Re: Building Effective "Agents"
#22When thinking about AI agents, there is still conflation between how to decide the next step to take vs what information is needed to decide the next step.
If runtime information is insufficient, we can use AI/ML models to fill that information. But deciding the next step could be done ahead of time assuming complete information.
Most AI agent examples short circuit these two steps. When faced with unstructured or insufficient information, the program asks the LLM/AI model to decide the next step. Instead, we could ask the LLM/AI model to structure/predict necessary information and use pre-defined rules to drive the process.
This approach will translate most [1] "Agent" examples into "Workflow" examples. The quotes here are meant to imply Anthropic's definition of these terms.
[1] I said "most" because there might be continuous world systems (such as real world simulacrum) that will require a very large number of rules and is probably impractical to define each of them. I believe those systems are an exception, not a rule.
Re: Building Effective "Agents"
#23Claude api lacks structured output, without uniformity in output, it's not useful as agent. I've had agents system broke down suddenly due to degradation in output, which leads to the previous suggested json output hacks (from official cookbook) stopped working.
Re: Building Effective "Agents"
#24Have been building agents for past 2 years, my tl;dr is that: Agents are Interfaces, Not Implementations The current zeitgeist seems to think of agents as passthrough agents: e.g. a lite wrapper around a core that's almost 100% a LLM. The most effective agents I've seen, and have built, are largely traditional software engineering with a sprinkling of LLM calls for "LLM hard" problems. LLM hard problems are problems…
Same experience.
The smaller and more focused the context, the higher the consistency of output, and the lower the chance of jank.
Fundamentally no different than giving instructions to a junior dev. Be more specific -- point them to the right docs, distill the requirements, identify the relevant areas of the source -- to get good output.
My last attempt at a workflow of agents was at the 3.5 to 4 transition and OpenAI wasn't good enough at that point to produce consistently good output and was slow to boot.
My team has taken the stance that getting consistently good output from LLMs is really an ETL exercise: acquire, aggregate, and transform the minimum relevant data for the output to reach the desired level of quality and depth and let the LLM do it's thing.
Re: Building Effective "Agents"
#25This is by far the most practical piece of writing I've seen on the subject of "agents" - it includes actionable definitions, then splits most of the value out into "workflows" and describes those in depth with example applications. There's also a cookbook with useful code examples: https://github.com/anthropics/anthropic-cookbook/tree/main/p... Blogged about this here: https://simonwillison.net/2024/Dec/20/building-…
I'm glad they are publishing their cookbooks recipes on github too. Openai used to be more active there.
[flagged]
Re: Building Effective "Agents"
#26My personal view is that the roadmap to AGI requires an LLM acting as a prefrontal cortex: something designed to think about thinking. It would decide what circumstances call for double-checking facts for accuracy, which would hopefully catch hallucinations. It would write its own acceptance criteria for its answers, etc. It's not clear to me how to train each of the sub-models required, or how big (or small!) they n…
After I read attention is all you need, my first thought was: "Orchestration is all you need". When 4o came out I published this: https://b.h4x.zip/agi/
Re: Building Effective "Agents"
#27Agents are still a misaligned concept in AI. While this article offers a lot in orchestration, memory (only mentioned once in the post) and governance are not really mentioned. The latter is important to increase reliability -- something Ilya Sutskever mentioned to be important as agents can be less deterministic in their responses.
Interestingly, "agency" i.e., the ability of the agent to make own decisions is not mentioned once.
I work on CAAs and document my journey on my substack (https://jdsmerau.substack.com)
Re: Building Effective "Agents"
#28Have been building agents for past 2 years, my tl;dr is that: Agents are Interfaces, Not Implementations The current zeitgeist seems to think of agents as passthrough agents: e.g. a lite wrapper around a core that's almost 100% a LLM. The most effective agents I've seen, and have built, are largely traditional software engineering with a sprinkling of LLM calls for "LLM hard" problems. LLM hard problems are problems…
There’ll always be an advantage for those who understand the problem they’re solving for sure. The balance of traditional software components and LLM driven components in a system is an interesting topic - I wonder how the capabilities of future generations of foundation model will change that?
Certain the end state is "one model to rule them all" hence the "transitional."
Just that the pragmatic approach, today, given current LLM capabilities, is to minimize the surface area / state space that the LLM is actuating. And then gradually expand that until the whole system is just a passthrough. But starting with a passthrough kinda doesn't lead to great products in December 2024.
Re: Building Effective "Agents"
#29Earlier quoted context omitted.
I'm glad they are publishing their cookbooks recipes on github too. Openai used to be more active there.
[flagged]
Eh, let's nip this in the bud: we could end up in a "it feels like...", coupled to free association, cycle. :)
More substantively, we can check our vibe. OpenAI is just as active as it ever was w/notebooks. To an almost absurd degree. 5-10 commits a week. https://github.com/openai/openai-cookbook/activity
Re: Building Effective "Agents"
#30Good article. I think it can emphasize a bit more on supporting human interactions in agentic workflows. While composing workflows isn't new, involving human-in-the-loop introduces huge complexity, especially for long-running, async processes. Waiting for human input (which could take days), managing retries, and avoiding errors like duplicate refunds or missed updates require careful orchestration.
I think this is where durable execution shines. By ensuring every step in an async processing workflow is fault-tolerant and durable, even interruptions won't lose progress. For example, in a refund workflow, a durable system can resume exactly where it left off—no duplicate refunds, no lost state.