Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Effective "Agents"

anthropic.com

Post reply on HN
Print view View original

51–60 of 130 posts

Re: Building Effective "Agents"

#51

Post by melvinmelih »

While I agree with the premise of keeping it simple (especially when it comes to using opaque and overcomplicated frameworks like LangChain/LangGraph!) I do believe there’s a lot more to building agentic systems than this article covers.

I recently wrote[1] about the 4 main components of autonomous AI agents (Profile, Memory, Planning & Action) and all of that can still be accomplished with simple LLM calls, but there’s simply a lot more to think about than simple workflow orchestration if you are thinking of building production-ready autonomous agentic systems.

[1] https://melvintercan.com/p/anatomy-of-an-autonomous-ai-agent

Re: Building Effective "Agents"

#52

Post by solidasparagus »

Animats wrote: post #50
simonw wrote: post #11

This is by far the most practical piece of writing I've seen on the subject of "agents" - it includes actionable definitions, then splits most of the value out into "workflows" and describes those in depth with example applications. There's also a cookbook with useful code examples: https://github.com/anthropics/anthropic-cookbook/tree/main/p... Blogged about this here: https://simonwillison.net/2024/Dec/20/building-…

Yes, they have actionable definitions, but they are defining something quite different than the normal definition of an "agent". An agent is a party who acts for another. Often this comes from an employer-employee relationship. This matters mostly when things go wrong. Who's responsible? The airline whose AI agent gave out wrong info about airline policies found, in court, that their "intelligent agent" was considere…

That's only one of many definitions for the word agent outside of the context of AI. Another is something produces effects on the world. Another is something that has agency.

Sort of interesting that we've coalesced on this term that has many definitions, sometimes conflicting, but where many of the definitions vaguely fit into what an "AI Agent" could be for a given person.

But in the context of AI, Agent as Anthropic defines it is an appropriate word because it is a thing that has agency.

Re: Building Effective "Agents"

#53

Post by debois »

Note how much the principles here resemble general programming principles: keep complexity down, avoid frameworks if you can, avoid unnecessary layers, make debugging easy, document, and test.

It’s as if AI took over the writing-the-program part of software engineering, but sort of left all the rest.

Re: Building Effective "Agents"

#54

Post by zby »

zby wrote: post #9
timdellinger wrote: post #5

My personal view is that the roadmap to AGI requires an LLM acting as a prefrontal cortex: something designed to think about thinking. It would decide what circumstances call for double-checking facts for accuracy, which would hopefully catch hallucinations. It would write its own acceptance criteria for its answers, etc. It's not clear to me how to train each of the sub-models required, or how big (or small!) they n…

IMHO with a simple loop LLMs are already capable of some meta thinking, even without any internal new architectures. For me where it still fails is that LLMs cannot catch their own mistakes even some obvious ones. Like with GPT 3.5 I had a persistent problem with the following question: "Who is older, Annie Morton or Terry Richardson?". I was giving it Wikipedia and it was correctly finding out the birth dates of the…

Ah yeah - actually I tested that taking out of context. This is the thing that surprised me - I thought it is about 'writing itself into a corner - but even in a completely different context the LLM is consistently doing an obvious mistake. Here is the example: https://chatgpt.com/share/67667827-dd88-8008-952b-242a40c2ac...

Janet Waldo was playing Corliss Archer on radio - and the quote the LLM found in Wikipedia was confirming it. But the question was about film - and the LLM cannot spot the gap in its reasoning - even if I try to warn it by telling it the report came from a junior researcher.

Re: Building Effective "Agents"

#55

Post by simonw »

Animats wrote: post #50
simonw wrote: post #11

This is by far the most practical piece of writing I've seen on the subject of "agents" - it includes actionable definitions, then splits most of the value out into "workflows" and describes those in depth with example applications. There's also a cookbook with useful code examples: https://github.com/anthropics/anthropic-cookbook/tree/main/p... Blogged about this here: https://simonwillison.net/2024/Dec/20/building-…

Yes, they have actionable definitions, but they are defining something quite different than the normal definition of an "agent". An agent is a party who acts for another. Often this comes from an employer-employee relationship. This matters mostly when things go wrong. Who's responsible? The airline whose AI agent gave out wrong info about airline policies found, in court, that their "intelligent agent" was considere…

Where did you get the idea that your definition there is the "normal" definition of agent, especially in the context of AI?

I ask because you seem very confident in it - and my biggest frustration about the term "agent" is that so many people are confident that their personal definition is clearly the one everyone else should be using.

Re: Building Effective "Agents"

#56

Post by zby »

My wish list for LLM APIs to make them more useful for 'agentic' workflows:

Finer grained control over the tools the LLM is supposed to use. The 'tool_choice' should allow giving a list of tools to choose. The point is that the list of all available tools is needed to interpret the past tool calls - so you cannot use it to also limit the LLM choice at a particular step. See also: https://zzbbyy.substack.com/p/two-roles-of-tool-schemas

Control over how many tool calls can go in one request. For stateful tools multiple tool calls in one request leads to confusion.

By the way - is anyone working with stateful tools? Often they seem very natural and you would think that the LLM at training should encounter lots of stateful interactions and be skilled in using them. But there aren't many examples and the libraries are not really geared towards that.

Re: Building Effective "Agents"

#57

Post by zaran »

This was an excellent writeup - felt a bit surprised at how much they considered "workflow" instead of agent but I think it's good to start to narrow down the terminology

I think these days the main value of the LLM "agent" frameworks is being able to trivially switch between model providers, though even that breaks down when you start to use more esoteric features that may not be implemented in cleanly overlapping ways

Re: Building Effective "Agents"

#58

Post by Animats »

solidasparagus wrote: post #52
Animats wrote: post #50

Earlier quoted context omitted.

Yes, they have actionable definitions, but they are defining something quite different than the normal definition of an "agent". An agent is a party who acts for another. Often this comes from an employer-employee relationship. This matters mostly when things go wrong. Who's responsible? The airline whose AI agent gave out wrong info about airline policies found, in court, that their "intelligent agent" was considere…

That's only one of many definitions for the word agent outside of the context of AI. Another is something produces effects on the world. Another is something that has agency. Sort of interesting that we've coalesced on this term that has many definitions, sometimes conflicting, but where many of the definitions vaguely fit into what an "AI Agent" could be for a given person. But in the context of AI, Agent as Anthrop…

> But in the context of AI, Agent as Anthropic defines it is an appropriate word because it is a thing that has agency.

That seems circular.

Re: Building Effective "Agents"

#60

Post by tlarkworthy »

th0ma5 wrote: post #36
simonw wrote: post #11

This is by far the most practical piece of writing I've seen on the subject of "agents" - it includes actionable definitions, then splits most of the value out into "workflows" and describes those in depth with example applications. There's also a cookbook with useful code examples: https://github.com/anthropics/anthropic-cookbook/tree/main/p... Blogged about this here: https://simonwillison.net/2024/Dec/20/building-…

How do you protect from compounding errors?

read the article, close the feedback loop with something verifiable (e.g. tests)
Post reply on HN