We decreased our LLM costs with Opus
mendral.com
We decreased our LLM costs with Opus
1–10 of 41 posts
Re: We decreased our LLM costs with Opus
#2> We switched to the "triager" pattern: a Haiku agent with a very specific and narrow job. Is this issue already tracked or not? If it is, stop right there. If not, escalate to Opus.
> 4 out of 5 failures never reach Opus. A triager match costs around 25x less than a full investigation.
The title feels misleading. Why clickbait on that when you can just be genuine about the architecture?
Re: We decreased our LLM costs with Opus
#3> We switched to the "triager" pattern: a Haiku agent with a very specific and narrow job. Is this issue already tracked or not? If it is, stop right there. If not, escalate to Opus. > 4 out of 5 failures never reach Opus. A triager match costs around 25x less than a full investigation. The title feels misleading. Why clickbait on that when you can just be genuine about the architecture?
The title does not match the article title: “We Upgraded to a Frontier Model and Our Costs Went Down”.
Re: We decreased our LLM costs with Opus
#4I have rewritten the article to be slightly shorter:
“Let a cheap agent decide if the expensive one is needed.”
Re: We decreased our LLM costs with Opus
#5Looking at the diagram, is this seriously a case of replacing basic functional concepts like "write to clickhouse" or "have we seen this before" to a model? could those be actual function calls in some language?
just seems wasteful all around. having an agent in the critical path when a regular expression (or similar) could do just seems odd. yeah haiku is cheap but re.match() is cheaper.
Re: We decreased our LLM costs with Opus
#6I have rewritten the article to be slightly shorter: “Let a cheap agent decide if the expensive one is needed.”
[dead]
Re: We decreased our LLM costs with Opus
#7> We switched to the "triager" pattern: a Haiku agent with a very specific and narrow job. Is this issue already tracked or not? If it is, stop right there. If not, escalate to Opus. > 4 out of 5 failures never reach Opus. A triager match costs around 25x less than a full investigation. The title feels misleading. Why clickbait on that when you can just be genuine about the architecture?
The title does not match the article title: “We Upgraded to a Frontier Model and Our Costs Went Down”.
It’s still misleading, though.
Re: We decreased our LLM costs with Opus
#8I do a similar thing with a "planner agent" that uses the cheapest (I think it's using openai-gpt-5.2-mini or something at like 20 cents for 1M.) that more or less emits a plan name, task list and the task list has a recommended model in each task. It's not perfect, but many of our tasks are accomplished with lighter weight models. When doing code generation or fixing we upgrade to a more expensive model, planning and decisions are done more cheaply. Keep in mind the tasks are relatively constrained, so planning done with a cheap agent makes sense here. An open-ended agent would likely use a more expensive call for planning.
Re: We decreased our LLM costs with Opus
#9I have rewritten the article to be slightly shorter: “Let a cheap agent decide if the expensive one is needed.”
Sounds like L1 vs L2 support :)
Re: We decreased our LLM costs with Opus
#10The whole clickbait article can be summarized in one line:
Let a cheap agent decide if the expensive one is needed