I am using my AI agents to optimize my agentic workflow. It is like "I am doing surgery on myself."
I burned all my tokens researching how to save tokens
111–120 of 237 posts
Re: I burned all my tokens researching how to save tokens
#112Earlier quoted context omitted.
Part of the issue with "shipping" AI-generated code is, what's the value of shipping something that your prospective customers could just make for themselves? Shipping a product used to be the gold standard of value, but I'm not sure it is anymore, the zeitgeist just yet has to catch up. I think it does have pretty good value when you are your own customer, because you can make exactly what you need. For instance, I'…
Most people still prefer products over building their own thing, even if the barrier is lower. As an example consumer 3D printing hasn’t had much impact on manufacturing
This is a bit of a year-of-linux-on-the-desktop question.
Desktop 3D printers are tools; I know many people using them to support their making. And they've utterly changed manufacturing on another level (simply because of their ability to surface a prototype).
It will get better and better and better and still never replace mass production, but mass production will be changed by it.
Not least because of the fundamental shift in availability of the tools to make tools. The productised desktop CNC market is wild and it owes almost its entire existence to the change in the market driven by 3D printing.
Re: I burned all my tokens researching how to save tokens
#113Earlier quoted context omitted.
Yeah it's very possible I hit a rail, as one of the projects involves fully automating Claude Code to allow co-driving (similar to what mobile remote control offers, but other agents (eg. Pi) can also be in control, and vice versa). And yes it does seem as though the token burn is also higher, even though it eventually outputs something that works). I'm a patient person though, as long as the rails aren't too restric…
Id recommend getting and running Qwen-3.5-35B-abliterated locally. Having a local LLM is immensely helpful especially if you do the thing the companies disapprove of. Abliteration guarantees it wont refuse or censor. The token dealers are incentivised in burning as much tokens as possible, and scamming by crippling/downgrading models. Their goal is to get you to pay more for each pull of the slot machine. And this sl…
Oh yes that reminds me, I have a few small local models including Qwen3.5. The tps rate is pretty bad for interactive use but I should definitely try it for my more automated workflows. I already use DeepSeek (via OR) alongside though and I doubt any Chinese provider will be restrictive in that way, so let Anthropic do what they may :).
Re: I burned all my tokens researching how to save tokens
#114Earlier quoted context omitted.
https://github.com/ByteTerrace/Puck/blob/main/src/Puck.World... "Shipped" is a bit strong in my case, but I think this counts as something that isn't mere slop.
One sentence into that readme I can tell you that yes, it most definitely counts as slop.
Jesus Wept. Software engineering is dying at the moment and everyone has AI Glitz in their eyes.
Re: I burned all my tokens researching how to save tokens
#115Earlier quoted context omitted.
One sentence into that readme I can tell you that yes, it most definitely counts as slop.
Jesus Wept. Software engineering is dying at the moment and everyone has AI Glitz in their eyes.
Dying? Really?
https://github.com/ByteTerrace/Puck/blob/main/src/Puck.Maths...
Some of us aren't just slopping it up like @theo and all the other AI influencers are, as I developed nearly every single line of this by hand over the span of the past decade. Agents recently became capable of factchecking it and helped find bugs that my other unit testing never caught. Software engineering has never been more alive!
https://gist.github.com/Kittoes0124/6827d08e457c1c8b790422af...
This is a snippet I wrote years ago to help me study different formulations of the equation ((1 + √5) / 2). There was this thing I could "see" in there that would allow me index into these sequences, and whole families of others, in O(1) time. The problem is that I still don't know enough maths to derive the exact object that I want from the nonsense that I have.
https://github.com/ByteTerrace/Puck/blob/main/src/Puck.Maths...
This is the reformulation of that snippet into a proper object, distilled using Fable and Sol. Unlike BinaryIntegerFunctions, I didn't write a line of this and I'm quite happy with that. It's been a slog for so long... being able to let these agents toil away at all the off-by-one type noise is just nice. I get to focus on an actual product for once instead of manufacturing every single nut and bolt myself.
Re: I burned all my tokens researching how to save tokens
#116Earlier quoted context omitted.
Trust me there are plenty of us using cloud AI to actually ship stuff. We just aren't writing blog posts about it.
what did you ship?
Most of "What are you working on?" is AI assisted; not much is shipped.
https://hn.algolia.com/?query=working%20author%3Adavid927&so...
Re: I burned all my tokens researching how to save tokens
#117Earlier quoted context omitted.
Well yeah, it'd require the models to be loaded on the same system and the cache to be shared between them somehow
Cache sharing is not possible. The numbers in the cache are completely specific to the model.
Currently. I'm sure that you could make a system where the cache values are a superset C of e.g. models A and B where C is probably bigger than max(A,B) but smaller than A+B
Re: I burned all my tokens researching how to save tokens
#118Earlier quoted context omitted.
That's what I have been doing with Cursor with auto mode. Let the model manage the cache through compaction but staying focused on the topic. It has been quite efficient cost wise and quality wise too. It found issuesbat cheaper cost that Sonnet 4.6 medium effort had missed while costing more. I would like to repeat this test with higher tier Anthropic models like Opus effort medium to see the results.
With a coding plan, there is no need to use lower quality models right (GPT 5.6, Fable 5). Only with some kind of batch job (processing PDFs) I could imagine not having enough usage with a 100 or 200 USD coding plan. It sounds like you're on API? I'm dying to know why, since it seems quite common, yet it's about ten times the cost. Sounds like a terrible deal. What am I missing?
What I said was specific to work and it is not entirely clear to me why the cost goes up so much the moment I switch to medium with just Sonnet 4.6 (newer models would be even more I guess). There is a monetary budget allocated and I've blown it easily in the past with Claude. Cursor has been a life saver.
Re: I burned all my tokens researching how to save tokens
#119Earlier quoted context omitted.
Couldn't someone build an adaptive system, where the llm is frequently judging the difficulty of a task and switching to a more/less powerful model?
The judgment of task difficulty is a cursed problem. In most domains you really don't know what is down there until you start digging a little bit.
Well yeah. You start on as medium model, it does the digging says says 'this task is too easy/hard/just right for me' and go from there.
Re: I burned all my tokens researching how to save tokens
#120Earlier quoted context omitted.
Yeah, that was pretty much my experience. The models weren’t lacking knowledge as much as discipline. Without a good workflow, they will most likely spend thousands of tokens exploring dead ends.
I think 'exploring the dead ends' is *possibly* very fruitful. Various disciplines have a lot of appearing dead ends that someone in another field did solve. And they don't talk with each other. What I've seen with lots of the breathless 'OMG SCIENCE ADVANCEMENT' articles including the one yesterday, is that the LLMs are quite extraordinary about linking a dozen different fields together, and delivering an answer com…
Agreed. My goal is not fewer dead ends, but fewer repeated dead ends.