Code mode yields a 99.2% cost reduction in our systems
agent-swarm.dev
Code mode yields a 99.2% cost reduction in our systems
1–10 of 77 posts
Re: Code mode yields a 99.2% cost reduction in our systems
#2Hi there!
We've been building https://github.com/desplega-ai/agent-swarm in the open for a while, and one of the things we wanted to do since the start is find how powerful it can be to push for a code mode like env for the swarm.
We tried it and we managed to reduce up +90% of token costs by using swarm scripts (the code mode variant we implemented) for some of our schedules.
This is game changer, as we are able to
1. Run those schedules faster 2. Cheaper 3. More reliably
Been thinking a lot now on how we should prompt and change the default templates to force the agents to build on top of this. If it can be re-used or it's a recurring thing -> scripts.
Thoughts? Have you seen this type of improvements in your setups?
Cheers,
Re: Code mode yields a 99.2% cost reduction in our systems
#3[deleted]
Re: Code mode yields a 99.2% cost reduction in our systems
#4I am still a bit surprised that this is not more widely used. Normal inference + tool calls is effectively composition via serialized natural language, whereas Code Mode / Programmatic Tool Calling is using purpose built languages (aka programming languages) for composition and concurrency. It somewhat feels obvious to me that the latter is way more token and latency efficient.
Re: Code mode yields a 99.2% cost reduction in our systems
#5A maxim I’ve arrived at working w llm every day is “put deterministic things in code, non deterministic in llm”. I do wish the harnesses would be more helpful in this regard. For example I’ve seen tons of scheduled jobs that people wrote in Claude/copilot/etc that could easily have been scripts. They aren’t scripts because the author doesn’t know how to script and they stop at the point the llm does what they want. It isn’t hard to then tell the llm “make a script to do all of the tool calls this prompt needs and then pass the result back to the llm for this non-deterministic part”.
Re: Code mode yields a 99.2% cost reduction in our systems
#6Hi there! We've been building https://github.com/desplega-ai/agent-swarm in the open for a while, and one of the things we wanted to do since the start is find how powerful it can be to push for a code mode like env for the swarm. We tried it and we managed to reduce up +90% of token costs by using swarm scripts (the code mode variant we implemented) for some of our schedules. This is game changer, as we are able to…
We didn't add this for the post. It's just what the swarm already does, and it's the same machinery behind Script Workflows.
What we hadn't done is measure it against Anthropic's own yardstick, with our own production data.
But the LLM that wrote your blog post says your system was already doing this and you just hadn't measured it...? How is it that you managed a reduction in token costs by changing nothing?
And can you provide a human-written explanation of the experimental methodology that gets you this miraculous, literally unbelievable 99.2% reduction in tokens?
Re: Code mode yields a 99.2% cost reduction in our systems
#7The article is so painful to read.
Re: Code mode yields a 99.2% cost reduction in our systems
#8I am still a bit surprised that this is not more widely used. Normal inference + tool calls is effectively composition via serialized natural language, whereas Code Mode / Programmatic Tool Calling is using purpose built languages (aka programming languages) for composition and concurrency. It somewhat feels obvious to me that the latter is way more token and latency efficient.
In the world of AI hype, everything is to be learned again and we are going to see many many breakthroughs like this. You can see this shift already in the tone of conversation in HN for example.
The basics of computer engineering is now novel approaches. We are discovering why programming languages exist, starting with a worst version of Cobal running on a very expensive and probabilistic VM.
We will shoehorn every square, rectangular, and all forms of peg through the AI hole.
Re: Code mode yields a 99.2% cost reduction in our systems
#9This quarter’s clankerisms: “Honest” and “vibes”.
The tech here is cool; good reminder to push for more scripts in general from the agentic tools.
Re: Code mode yields a 99.2% cost reduction in our systems
#10codex CLI is already leveraging this internally. instead of performing raw MCP calls its internally using a js REPL to do that. elicitation is then propagated for approvals independently / it's interpreted as regular tool calls for operators etc.
so if i were to tell an agent to move the contents of a confluence doc into a file, it would do so without even reading the confluence page - in theory.