Has anyone got an example of an agent doing work in production that is saving the company money and doing a genuinely worthwhile job (in other words it's not writing text that exists purely to fill space on a packet of chips)?
Building Effective AI Agents
71–80 of 93 posts
Re: Building Effective AI Agents
#72Earlier quoted context omitted.
They mentioned hand offs, sub agents, concurrent tool calls, etc. You could write that yourself, but you would be inventing your own framework.
Sub-agent is another LLM loop that you simply import and provide as a tool to your orchestrator LLM. For example in Claude Code, sub-agent is a tool called "Task( )" made available to the main LLM (the one that you chat with) along with other tools like patch_file and web_search. Concurrent tool call is when LLM writes multiple tool calls instead of one, and you can program your app to execute those sequentially or c…
> The "agent framework" layer here is so thin it might as well don't exist
There's plenty of things that you need to make an AI agent that I woudn't want to re-implement or copy and paste each time. The most annoying being automatic conversation history summarization (e.g. I accidentally wasted $60 with the latest OpenAI realtime model, because the costs go up very quickly as the conversation history grows). And I'm sure we'll discover more things like that in the future.
Re: Building Effective AI Agents
#73Earlier quoted context omitted.
I believe the definition of workflows in this article is inaccurate. Workflows in modern engines do not take predefined code paths, and agents are effectively the same as workflows in these cases. The redefinition of workflows seems to be an attempt to differentiate, but for the most part an agent is nothing more than a workflow that is a loop that dynamically invokes things based on LLM responses. Modern workflow en…
I think the distinction is more about the "level of railroading". Workflows have a lot more structure and rules about information and control flow. Agents, on the other hand, are often given a set of tools and a prompt. They are much more free-form. For example, a workflow might define a fuzzy rule like "if customer issue is refund, go to refund flow," while an agent gets customer service tools and figures out how to…
Just to emphasize your point, below is a workflow I wrote for an LLM recently, to do language tagging (e.g., of vocab, grammar structures, etc). It's very different than what you'd think of as an "agent", where the LLM has tools and can take initiative.
LLMs are amazingly powerful in some ways, but without this kind of "scaffolding", simply not reliable enough to make consistent choices.
---
1. Here are: a) a "language schema" describing what kinds of tags I want and why, with examples, b) The text I want you to tag c) A list of previously-defined tags which could potentially be relevant (simple string match)
List for yourself which pre-existing tags you plan to use when doing tagging.
[LLM generates a list of tags]
2. Here is a,b,c from above, and d) your own tag list
Please write a draft tag.
[LLM writes a draft]
3. Here is a-d from above, plus e) your first draft, and f) Some programmatically-generated "linter" warnings which may or may not be violations of the schema.
Please check over your draft to make sure it follows the schema.
[LLM writes a new draft]
Agent checks for "hard" rules, like making sure there's a 1-1 correlation between the text and the tags. If no rules are violated move to step 5.
4. Here is a-e from above, plus g) your most recent draft, and h) known rule violations. Please fix the errors.
[LLM writes a new draft]
Repeat 4 until no hard rules are broken.
5. [and so on]
Re: Building Effective AI Agents
#74As someone who works for a company having a real Agent in production, (not a workflow), I cannot disagree more than the very first statement here: Use Agent Frameworks like Langraph. We did exactly that, and had to throw everything away just a month down the line. Then we built everything from scratch and now our system scales pretty well. To be fair, I think there might be a space for using Agent Frameworks, but the…
It sounds like you're agreeing with the article? From TFA:
> Over the past year, we've worked with dozens of teams building large language model (LLM) agents across industries. Consistently, the most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns.
> ...There are many frameworks that make agentic systems easier to implement. ...These frameworks make it easy to get started by simplifying standard low-level tasks like calling LLMs, defining and parsing tools, and chaining calls together. However, they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug. They can also make it tempting to add complexity when a simpler setup would suffice. We suggest that developers start by using LLM APIs directly: many patterns can be implemented in a few lines of code.
Re: Building Effective AI Agents
#75Earlier quoted context omitted.
Sub-agent is another LLM loop that you simply import and provide as a tool to your orchestrator LLM. For example in Claude Code, sub-agent is a tool called "Task( )" made available to the main LLM (the one that you chat with) along with other tools like patch_file and web_search. Concurrent tool call is when LLM writes multiple tool calls instead of one, and you can program your app to execute those sequentially or c…
> The "agent framework" layer here is so thin it might as well don't exist There's plenty of things that you need to make an AI agent that I woudn't want to re-implement or copy and paste each time. The most annoying being automatic conversation history summarization (e.g. I accidentally wasted $60 with the latest OpenAI realtime model, because the costs go up very quickly as the conversation history grows). And I'm…
I would highly recommend gemini 2.5 pro too for their speech quality. It's priced lower and the quality is top notch on their API. I made an implementation here in case you're interested https://www.github.com/akdeb/ElatoAI but its on hardware so maybe not totally relevant
Re: Building Effective AI Agents
#76Earlier quoted context omitted.
> The "agent framework" layer here is so thin it might as well don't exist There's plenty of things that you need to make an AI agent that I woudn't want to re-implement or copy and paste each time. The most annoying being automatic conversation history summarization (e.g. I accidentally wasted $60 with the latest OpenAI realtime model, because the costs go up very quickly as the conversation history grows). And I'm…
I would highly recommend gemini 2.5 pro too for their speech quality. It's priced lower and the quality is top notch on their API. I made an implementation here in case you're interested https://www.github.com/akdeb/ElatoAI but its on hardware so maybe not totally relevant
I'm using LiveKit, and I indeed have tested Gemini, but it appears to be broken or at least incompatible with OpenAI. Not sure if this is a Livekit issue or a Gemini issue. Anyway I decided to go back to just using LLM, SST and TTS as separate nodes, but I've also been looking into Deepgram Voice Agent API, but LiveKit doesn't support it (yet?).
Re: Building Effective AI Agents
#77Earlier quoted context omitted.
Nothing works automagically. You still have to build in all the operational characteristics that you would for any traditional system. It's deceptively easy to look at some AI agent demos and think "oh, I can replace my team's huge mess of spaghetti code with a few clever AI prompts!" And it may even work for the first couple use cases. But all that code is there for a reason, and eventually it'll have to be reckoned…
Then wtf is the point of this?
That's the neat part - there is none!
Re: Building Effective AI Agents
#78As someone who works for a company having a real Agent in production, (not a workflow), I cannot disagree more than the very first statement here: Use Agent Frameworks like Langraph. We did exactly that, and had to throw everything away just a month down the line. Then we built everything from scratch and now our system scales pretty well. To be fair, I think there might be a space for using Agent Frameworks, but the…
What job is the agent performing?
Re: Building Effective AI Agents
#79Has anyone got an example of an agent doing work in production that is saving the company money and doing a genuinely worthwhile job (in other words it's not writing text that exists purely to fill space on a packet of chips)?
I like ChatIPT! It solves a real challenge with biodiversity data. It doesn't mention the term 'agentic' but there's definitely Python code being written and executed.
https://www.gbif.org/news/6aw2VFiEHYlqb48w86uKSf/chatipt-sys...
It's still in beta.
Press release:
Rukaya Johaadien's chatbot provides conversation-style support to students and researchers who hold biodiversity data but are first-time or infrequent data publishers. Its prompts guide users as it cleans and standardizes spreadsheets, creates basic metadata, and publishes well-structured datasets on GBIF.org as a Darwin Core Archive.
To date, publishing high quality data from PhD and Master's degrees and other small-scale biodiversity research studies has been difficult to do at scale. Standardizing data typically requires specialist knowledge of programming languages, data management techniques, and familiarity with specialist software.
Meanwhile, the process of gaining access to existing instances of the Integrated Publishing Toolkit (IPT)—the GBIF network's workhorse application for data sharing run by node staff with limited time and resources—can test a novice's patience. Training can do little to surmount such logistical barriers and others, like language, when occasional users forget the precise steps and details from year to year.
"Data standardization is hard, and biologists don't become biologists because they like coding or Excel, so a lot of potentially valuable data falls by the wayside," said Johaadien. "Recognizing that large language models have gotten really good at generating code and working with data, I built an automated tool to guide non-technical users through routine questions and process their messy data as much as possible, then publish it quickly and automatically to GBIF."
Re: Building Effective AI Agents
#80This article remains one of the better pieces on this topic, especially since it clearly defines which definition of "AI agents" they are using at the start! They use: "systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks". I also like the way they distinguish between "agents" and "workflows", and describe a bunch of useful workflow patterns. I p…
The article on the multi-agent research is awesome. I do disagree with one statement in the building effective AI agents article - building your initial system without a framework sounds nice as an educational endeavor but the first benefit you get from a good framework is the easy ability to try out different (and cross-vendor) LLMs
not only that it also ready you for production if the framework has constructs like observability, eval, deployment, cloud security , ect...