Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Effective "Agents"

anthropic.com

Post reply on HN
Print view View original

61–70 of 130 posts

Re: Building Effective "Agents"

#61

Post by ankit219 »

jascha_eng wrote: post #2

I put the agents in quotes because anthropic actually talks more about what they call "workflows". And imo this is where the real value of LLMs currently lies, workflow automation. They also say that using LangChain and other frameworks is mostly unnecessary and does more harm than good. They instead argue to use some simple patterns, directly on the API level. Not dis-similar to the old-school Gang of Four software…

Deploying in production, the current agentic systems do not really work well. Workflow automation does. The reason is very native to LLMs, but also incredibly basic. Every agentic system starts with planning and reasoning module, where an LLM evaluates the task given and plans about how to accomplish that task, before moving on to next steps.

When an agent is given a task, they inevitably come up with different plans on different tries due to inherent nature of LLMs. Most companies like this step to be predictable, and they end up removing it from the system and doing it manually. Thus turning it into a workflow automation vs an agentic system. I think this is what people actually mean when they want to deploy agents in production. LLMs are great at automation*, not great at problem solving. Examples I have seen - customer support (you want predictability), lead mining, marketing copy generation, code flows and architecture, product specs generation, etc.

The next leap for AI systems is going to be whether they can solve challenging problems at companies - being the experts vs the doing the task they are assigned. They should really be called agents, not the current ones.

Re: Building Effective "Agents"

#64

Post by dmezzetti »

simonw wrote: post #11

This is by far the most practical piece of writing I've seen on the subject of "agents" - it includes actionable definitions, then splits most of the value out into "workflows" and describes those in depth with example applications. There's also a cookbook with useful code examples: https://github.com/anthropics/anthropic-cookbook/tree/main/p... Blogged about this here: https://simonwillison.net/2024/Dec/20/building-…

If you're looking for a lightweight open-source framework designed to handle the patterns mentioned in this article: https://github.com/neuml/txtai

Disclaimer: I'm the author of the framework.

Re: Building Effective "Agents"

#65

Post by shinryuu »

brotchie wrote: post #10

Have been building agents for past 2 years, my tl;dr is that: Agents are Interfaces, Not Implementations The current zeitgeist seems to think of agents as passthrough agents: e.g. a lite wrapper around a core that's almost 100% a LLM. The most effective agents I've seen, and have built, are largely traditional software engineering with a sprinkling of LLM calls for "LLM hard" problems. LLM hard problems are problems…

Do you have a public example of a good agentic system. I would like to experience it.

Re: Building Effective "Agents"

#66

Post by PhilippGille »

simonw wrote: post #55
Animats wrote: post #50

Earlier quoted context omitted.

Yes, they have actionable definitions, but they are defining something quite different than the normal definition of an "agent". An agent is a party who acts for another. Often this comes from an employer-employee relationship. This matters mostly when things go wrong. Who's responsible? The airline whose AI agent gave out wrong info about airline policies found, in court, that their "intelligent agent" was considere…

Where did you get the idea that your definition there is the "normal" definition of agent, especially in the context of AI? I ask because you seem very confident in it - and my biggest frustration about the term "agent" is that so many people are confident that their personal definition is clearly the one everyone else should be using.

Didn't he mention it was the court's definition?

But I'm not sure if that's true. The court didn't define anything, in contrary they only said that (in simplified terms) the chatbot was part of the website and it's reasonable to expect the info on their website to be accurate.

The closest I could find to the chatbot being considered an agent in legal terms (an entity like an employee) is this:

> Air Canada argues it cannot be held liable for information provided by one of its agents, servants, or representatives – including a chatbot.

Source: https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/202...

Re: Building Effective "Agents"

#67

Post by JonChesterfield »

simonw wrote: post #55
Animats wrote: post #50

Earlier quoted context omitted.

Yes, they have actionable definitions, but they are defining something quite different than the normal definition of an "agent". An agent is a party who acts for another. Often this comes from an employer-employee relationship. This matters mostly when things go wrong. Who's responsible? The airline whose AI agent gave out wrong info about airline policies found, in court, that their "intelligent agent" was considere…

Where did you get the idea that your definition there is the "normal" definition of agent, especially in the context of AI? I ask because you seem very confident in it - and my biggest frustration about the term "agent" is that so many people are confident that their personal definition is clearly the one everyone else should be using.

Defining "agent" as "thing with agency" seems legitimate to me, what with them being the same word.

Re: Building Effective "Agents"

#68

Post by Kydlaw »

jascha_eng wrote: post #2

I put the agents in quotes because anthropic actually talks more about what they call "workflows". And imo this is where the real value of LLMs currently lies, workflow automation. They also say that using LangChain and other frameworks is mostly unnecessary and does more harm than good. They instead argue to use some simple patterns, directly on the API level. Not dis-similar to the old-school Gang of Four software…

In fact they are mentioning LangGraph (the agent framework from the LangChain company). Imo LangGraph is a much more thoughtful and better built piece of software than the LangChain framework.

As I said, they already mention LangGraph in the article, so the Anthropic's conclusions still hold (i.e. KISS).

But this thread is going in the wrong direction when talking about LangChain

Re: Building Effective "Agents"

#69

Post by Nevermark »

Animats wrote: post #58
solidasparagus wrote: post #52

Earlier quoted context omitted.

That's only one of many definitions for the word agent outside of the context of AI. Another is something produces effects on the world. Another is something that has agency. Sort of interesting that we've coalesced on this term that has many definitions, sometimes conflicting, but where many of the definitions vaguely fit into what an "AI Agent" could be for a given person. But in the context of AI, Agent as Anthrop…

> But in the context of AI, Agent as Anthropic defines it is an appropriate word because it is a thing that has agency. That seems circular.

It would only be circular if agency was only defined as “the property of being an agent”. That circle of reasoning isn’t being proposed as the formal definitions by anyone.

Perhaps you mean tautological. In which case, an agent having agency would be an informal tautology. A relationship so basic to the subject matter that it essentially must be true. Which would be the strongest possible type of argument.

Re: Building Effective "Agents"

#70

Post by adeptima »

The whole Agent thing can easily blow in complexity.

Here some challenges I personally faced recently

- Durable Execution Paradigm: You may need the system to operate in a "durable execution" fashion like Temporal, Hatchet, Inngest, and Windmill. Your processes need to run for months, be upgraded and restarted. Links below

- FSM vs. DAG: Sometimes, a Finite State Machine (FSM) is more appropriate than a Directed Acyclic Graph (DAG) for my use cases. FSMs support cyclic behavior, allowing for repeated states or loops (e.g., in marketing sequences). FSM done right is hard. If you need FSM, you can't use most tools without "magic" hacking

- Observability and Tracing - takes time to put it everything nice in Grafana (Alloy, Tempo, Loki, Prometheus) or whatever you prefer. Attention switch between multiple systems is not an option during to limited attention span and "skills" issue. Most of "out of box" functionality or new Agents frameworks quickly becomes a liability

- Token/Inference Economy - token consumption and identifying edge cases with poor token management is a challenge, similar to Ethereum's gas consumption issues. Building a billing system based on actual consumption on the top of Stripe was a challenge. This is even 10x harder ... at least for me ;)

- Context Switching - managing context switching is akin to handling concurrency and scheduling with async/await paradigms, which can become complex. Simple prompts is a ok, but once you start joggling documents or screenshots or screen reading it's another game.

What I like about the all above it's nothing new - all design patterns, architecture are known for a while.

It's just hard to see it through AI/ML buzzwords storm ... but once you start looking at source code ... the fog of mind wars become clear.

Durable Execution / Workflow Engines

- Temporal https://github.com/temporalio - https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

- Hatchet https://news.ycombinator.com/item?id=39643136

- Inngest https://news.ycombinator.com/item?id=36403014

- Windmill https://news.ycombinator.com/item?id=35920082

Any comments and links on the above challenges and solutions are greatly appreciated!

Post reply on HN