I have always voted for the Unix style multiple do one thing good blackboxes as the plumbing in the ruling agent.
Divide and conquer me hearties.
Building Effective "Agents"
41–50 of 130 posts
Re: Building Effective "Agents"
#42Re: Building Effective "Agents"
#43> Agents can be used for open-ended problems where it’s difficult or impossible to predict the required number of steps, and where you can’t hardcode a fixed path. The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making. Agents' autonomy makes them ideal for scaling tasks in trusted environments. The questions then become: 1. When can you (i.e. a person who wants…
Just take any example and think how a human would break it down with decision trees. You are building an AI system to respond to your email. The first agent decides whether the new email should be responded to, yes or no. If no, it can send it to another LLM call that decides to archive it or leave it in the inbox for the human. If yes, it sends it to classifier that decides what type of response is required. Maybe t…
> The first agent decides whether the new email should be responded to, yes or no.
How would you trust that the agent is following the criteria, and how sure that the criteria is specific enough. Like someone you just meet told you they going to send you something via email, but then the agent misinterpret it due to missing context and decided to respond in a generic manner leading to misunderstanding.
> assume every single LLM call has been hijacked and don’t trust its input/output and you’ll be good.
Which is not new. But with formal languages, you have a more precise definition of what acceptable inputs are (the whole point of formalism is precise definitions). With LLM workflows, the whole environment should be assumed to be public information. And you should probably add a fine point that the output does not engage you in anything.
Re: Building Effective "Agents"
#44My personal view is that the roadmap to AGI requires an LLM acting as a prefrontal cortex: something designed to think about thinking. It would decide what circumstances call for double-checking facts for accuracy, which would hopefully catch hallucinations. It would write its own acceptance criteria for its answers, etc. It's not clear to me how to train each of the sub-models required, or how big (or small!) they n…
IMHO with a simple loop LLMs are already capable of some meta thinking, even without any internal new architectures. For me where it still fails is that LLMs cannot catch their own mistakes even some obvious ones. Like with GPT 3.5 I had a persistent problem with the following question: "Who is older, Annie Morton or Terry Richardson?". I was giving it Wikipedia and it was correctly finding out the birth dates of the…
Brains are split internally, with each having their own monologue. One happens to have command.
Re: Building Effective "Agents"
#45Earlier quoted context omitted.
IMHO with a simple loop LLMs are already capable of some meta thinking, even without any internal new architectures. For me where it still fails is that LLMs cannot catch their own mistakes even some obvious ones. Like with GPT 3.5 I had a persistent problem with the following question: "Who is older, Annie Morton or Terry Richardson?". I was giving it Wikipedia and it was correctly finding out the birth dates of the…
Brains are split internally, with each having their own monologue. One happens to have command.
I don't think there's reason to believe both halves have a monologue, is there? Experience, yes, but doesn't only one half do language?
Re: Building Effective "Agents"
#46Does any one have a solid examples of a real agent, deployed in production?
Re: Building Effective "Agents"
#47Earlier quoted context omitted.
Brains are split internally, with each having their own monologue. One happens to have command.
I don't think there's reason to believe both halves have a monologue, is there? Experience, yes, but doesn't only one half do language?
So if like me you have an interior dialogue, which is speaking and which is listening or is it the same one? I do not ascribe the speaker or listener to a lobe, but whatever the language and comprehension centre(s) is(are), it can do both at the same time.
Re: Building Effective "Agents"
#48Tangent but anyone know what software is used to draw those workflow diagrams?
Re: Building Effective "Agents"
#49Tangent but anyone know what software is used to draw those workflow diagrams?
they probably use their own designers. jk. the arrows looks a lot like Figma/Figjam.
Re: Building Effective "Agents"
#50This is by far the most practical piece of writing I've seen on the subject of "agents" - it includes actionable definitions, then splits most of the value out into "workflows" and describes those in depth with example applications. There's also a cookbook with useful code examples: https://github.com/anthropics/anthropic-cookbook/tree/main/p... Blogged about this here: https://simonwillison.net/2024/Dec/20/building-…
Yes, they have actionable definitions, but they are defining something quite different than the normal definition of an "agent". An agent is a party who acts for another.
Often this comes from an employer-employee relationship.
This matters mostly when things go wrong. Who's responsible? The airline whose AI agent gave out wrong info about airline policies found, in court, that their "intelligent agent" was considered an agent in legal terms. Which meant the airline was stuck paying for their mistake.
Anthropic's definition: Some customers define agents as fully autonomous systems that operate independently over extended periods, using various tools to accomplish complex tasks.
That's an autonomous system, not an agent. Autonomy is about how much something can do without outside help. Agency is about who's doing what for whom, and for whose benefit and with what authority. Those are independent concepts.