I am once again asking for people to ask Claude to reformat their article text to ASD-STE100 so that I don't have to read slop
will do next time, thanks!
Code mode yields a 99.2% cost reduction in our systems
31–40 of 77 posts
Re: Code mode yields a 99.2% cost reduction in our systems
#32This quarter’s clankerisms: “Honest” and “vibes”. The tech here is cool; good reminder to push for more scripts in general from the agentic tools.
thanks!
Re: Code mode yields a 99.2% cost reduction in our systems
#33I could be misreading this, but looking at the numbers, it seems that--despite the claim "we didn't estimate this"--basically everything in this is an estimate. It's a little hard to tell though, because the generated prose seems to occasionally contradict or repeat itself.
Re: Code mode yields a 99.2% cost reduction in our systems
#34you lose the per-call approvals though. one script hitting ten tools is harder to gate than ten separate calls
yes, that is true.
in fact we are thinking a lot about this, not on a toll based, but rather on a role based.
giving access to specific tools to specific agents, how that works and evolves, and how that links w the humans using it
Re: Code mode yields a 99.2% cost reduction in our systems
#35I could be misreading this, but looking at the numbers, it seems that--despite the claim "we didn't estimate this"--basically everything in this is an estimate. It's a little hard to tell though, because the generated prose seems to occasionally contradict or repeat itself.
I read it like that: we haven't measured this YET until now, and now we did
Re: Code mode yields a 99.2% cost reduction in our systems
#36The article is so painful to read.
Yes and it’s also something that was already known and recommended. It’s not “code mode” or something revolutionary, it’s just instructing your LLM to feel free to write scripts to aid it in completing the task. It’s very useful and I use it all the time, and in the end you can distill it into a reusable toolbox, somewhat similar to a collection of markdown files with project knowledge.
still most harnesses and AI systems do not use it, adopt it, or reference it enough...
Re: Code mode yields a 99.2% cost reduction in our systems
#37The article is so painful to read.
Because it was probably written by an LLM. Use a LLM to summarize it.
Relevant Slavoj Zizek talk: https://www.youtube.com/watch?v=qj4cxK-nOTw
Re: Code mode yields a 99.2% cost reduction in our systems
#38The problem is that when the LLM writes ad-hoc code how can you trust it? For example in my Liveclip MCP server I'm working on (not released yet) I have table manipulation type tools So for example a couple days ago Claude made a combined ranking score for my Youtube Shorts analytics -- it did all these tool calls in the same turn First it normalized the % values into numbers { "destination_col": "G", "key": "cinemas…
we have a way to see created scripts in a dashboard, and in fact they are not ad-hoc only, i.e. it can create "saved scripts" that it can re-use, iterate and so on. Also not exec happens in a sandboxed env, which means it might not have access to the fs by default (it has to agent-fs, which is a layer we built on top of s3 for agents)
Yeah I guess the dichotomy between 'tools' and 'code mode' isn't really as clear especially if the tools themselves are intelligent and truncate long results and have start_range, end_range type arguments. So the gist is about 'what part of the output needs to reach the LLM' and a 'code mode' approach can help orchestrate that just like good tools can
Re: Code mode yields a 99.2% cost reduction in our systems
#39A maxim I’ve arrived at working w llm every day is “put deterministic things in code, non deterministic in llm”. I do wish the harnesses would be more helpful in this regard. For example I’ve seen tons of scheduled jobs that people wrote in Claude/copilot/etc that could easily have been scripts. They aren’t scripts because the author doesn’t know how to script and they stop at the point the llm does what they want. I…
I’d expect the llm to write the script and run it.
The tooling can remain generic and token spending focused.
Re: Code mode yields a 99.2% cost reduction in our systems
#40I could be misreading this, but looking at the numbers, it seems that--despite the claim "we didn't estimate this"--basically everything in this is an estimate. It's a little hard to tell though, because the generated prose seems to occasionally contradict or repeat itself.
thanks for the feedback!
it's really hard to measure some times. for example, once of the things we have seen is that the specific `workflow-triage` schedule we have have been running much faster and cheaper since we moved some parts of it to use scripts. From our analytics we saw times from ~5min before to ~1 or 2 min now. And in terms of costs we saw +50% reduction.
Why not 90%? because even if it uses the script, that's the "repeatable" part. we still have a reasoning around the output of it, and then deciding how to escalate, which removes the 40% reduction. Still 50% is nice for something we run daily.