Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code mode yields a 99.2% cost reduction in our systems

agent-swarm.dev

Post reply on HN
Print view View original

21–30 of 77 posts

Re: Code mode yields a 99.2% cost reduction in our systems

#21

Post by jorisw »

somnium_sn wrote: post #4

I am still a bit surprised that this is not more widely used. Normal inference + tool calls is effectively composition via serialized natural language, whereas Code Mode / Programmatic Tool Calling is using purpose built languages (aka programming languages) for composition and concurrency. It somewhat feels obvious to me that the latter is way more token and latency efficient.

TFA suggests that it already is:

> The one-line takeaway

> This already ships by default in every claude/codex/opencode session's system prompt

Re: Code mode yields a 99.2% cost reduction in our systems

#22

Post by tarasyarema »

firasd wrote: post #15

The problem is that when the LLM writes ad-hoc code how can you trust it? For example in my Liveclip MCP server I'm working on (not released yet) I have table manipulation type tools So for example a couple days ago Claude made a combined ranking score for my Youtube Shorts analytics -- it did all these tool calls in the same turn First it normalized the % values into numbers { "destination_col": "G", "key": "cinemas…

we have a way to see created scripts in a dashboard, and in fact they are not ad-hoc only, i.e. it can create "saved scripts" that it can re-use, iterate and so on.

Also not exec happens in a sandboxed env, which means it might not have access to the fs by default (it has to agent-fs, which is a layer we built on top of s3 for agents)

Re: Code mode yields a 99.2% cost reduction in our systems

#23

Post by moberemk »

tarasyarema wrote: post #16
mpalmer wrote: post #6

Earlier quoted context omitted.

We didn't add this for the post. It's just what the swarm already does, and it's the same machinery behind Script Workflows. What we hadn't done is measure it against Anthropic's own yardstick, with our own production data. You say here that "since the start" you wanted to do a "code mode like env". But the LLM that wrote your blog post says your system was already doing this and you just hadn't measured it...? How i…

Hi, the blog post was written assisted with our own swarm, based on the experiment. Indeed we had the concept of workflows from the start (like n8n), but the point of the scripts is type safe executions that the agents can write, with access to APIs, MCPs, etc. E.g. this PR from ~10 days ago https://github.com/desplega-ai/agent-swarm/pull/934 was going in this direction, offering a way to defined type safe connection…

No offence but, “if you find yourself doing the same thing over and over again write a script for it” isn’t exactly novel thinking in the software field…this was the same process when it was a human doing it, we just measure that in time instead of tokens

Re: Code mode yields a 99.2% cost reduction in our systems

#25

Post by firasd »

asdfsa32 wrote: post #20
firasd wrote: post #15

The problem is that when the LLM writes ad-hoc code how can you trust it? For example in my Liveclip MCP server I'm working on (not released yet) I have table manipulation type tools So for example a couple days ago Claude made a combined ranking score for my Youtube Shorts analytics -- it did all these tool calls in the same turn First it normalized the % values into numbers { "destination_col": "G", "key": "cinemas…

A python script is now more iffy than a LLM going through JSON?

Yeah, of course. The MCP server tools do use Python in the backend but just look at the affordances it provided in the args -- named columns, a fixed match/replace API, a way to specify the range of rows. The LLM is not gonna create a little library with these affordances in an ephemeral REPL it's just gonna YOLO some pandas code that you hope did what you wanted and addressed the right columns, had the right types, etc

Re: Code mode yields a 99.2% cost reduction in our systems

#26

Post by tarasyarema »

AmazingTurtle wrote: post #10

codex CLI is already leveraging this internally. instead of performing raw MCP calls its internally using a js REPL to do that. elicitation is then propagated for approvals independently / it's interpreted as regular tool calls for operators etc. so if i were to tell an agent to move the contents of a confluence doc into a file, it would do so without even reading the confluence page - in theory.

Yes, that's the right direction codex is taking 100%

Now our idea is to offer this to be harness agnostic (we support codex, claude code, pi, opencode, even devin). And you (as the one deploying AI systems) should be in control of the AI, the execution and the data. That's our goal

Re: Code mode yields a 99.2% cost reduction in our systems

#27

Post by tarasyarema »

lyall wrote: post #17
tarasyarema wrote: post #2

Hi there! We've been building https://github.com/desplega-ai/agent-swarm in the open for a while, and one of the things we wanted to do since the start is find how powerful it can be to push for a code mode like env for the swarm. We tried it and we managed to reduce up +90% of token costs by using swarm scripts (the code mode variant we implemented) for some of our schedules. This is game changer, as we are able to…

There might be something valuable in this article, but if the you couldn't be bothered to write it yourself, why should we be bothered to read it?

This is based on a multi-day convo I had with our swarm. I might have not wrote it myself, but it's based on human interaction and thought process.

Any recommendations on how I could make it better next time, so you do not feel like wasting your time (which I get and feel, dont get me wrong!)

Re: Code mode yields a 99.2% cost reduction in our systems

#28

Post by [deleted] »

tarasyarema wrote: post #16
mpalmer wrote: post #6

Earlier quoted context omitted.

We didn't add this for the post. It's just what the swarm already does, and it's the same machinery behind Script Workflows. What we hadn't done is measure it against Anthropic's own yardstick, with our own production data. You say here that "since the start" you wanted to do a "code mode like env". But the LLM that wrote your blog post says your system was already doing this and you just hadn't measured it...? How i…

Hi, the blog post was written assisted with our own swarm, based on the experiment. Indeed we had the concept of workflows from the start (like n8n), but the point of the scripts is type safe executions that the agents can write, with access to APIs, MCPs, etc. E.g. this PR from ~10 days ago https://github.com/desplega-ai/agent-swarm/pull/934 was going in this direction, offering a way to defined type safe connection…

[deleted]

Re: Code mode yields a 99.2% cost reduction in our systems

#30

Post by tarasyarema »

moberemk wrote: post #23
tarasyarema wrote: post #16

Earlier quoted context omitted.

Hi, the blog post was written assisted with our own swarm, based on the experiment. Indeed we had the concept of workflows from the start (like n8n), but the point of the scripts is type safe executions that the agents can write, with access to APIs, MCPs, etc. E.g. this PR from ~10 days ago https://github.com/desplega-ai/agent-swarm/pull/934 was going in this direction, offering a way to defined type safe connection…

No offence but, “if you find yourself doing the same thing over and over again write a script for it” isn’t exactly novel thinking in the software field…this was the same process when it was a human doing it, we just measure that in time instead of tokens

yes, 100%

but it's not that simple. the whole point of this is that if you have an AI system that is in charge of this, what you need is a way for it to build those deterministic software based parts, while maintaining the AI routing and reasoning.

with time, our internal swarm has been building those "internal" software that is tedious and boring to build, where it makes sense.

e.g. we also have a "pages" thing (like claude arifacts) that it uses to join scripts, html and other components to build dashboards for us

Post reply on HN