Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I burned all my tokens researching how to save tokens

quesma.com

Post reply on HN
Print view View original

191–200 of 237 posts

Re: I burned all my tokens researching how to save tokens

#191

Post by a_c »

bkotrys wrote: post #184
a_c wrote: post #181

My own set of heuristics 1. Use less subagents. The act of using subagent already needs dumping adequate amount of context for subagent to work effectively 2. Refactor when file grows large. Reasonably scoped files is easier to understand, for both human and agents, and consume less tokens because less guessing which section of the file to read 3. More capable models for planning, learning. Cheap models for execution…

The audit point is especially interesting. Is your Claude token-audit tool available anywhere?

I didn't update it for a while. It reads your local conversation history, correlate input/output token with usage (claude doesn't give exact breakdown), and classify if they are subagent/tooluse/normal message/whatever. Let me know if I can make it any way useful to you.

https://github.com/kmcheung12/ccaudit

Re: I burned all my tokens researching how to save tokens

#192

Post by ncphillips »

dominotw wrote: post #30
resonious wrote: post #20

Earlier quoted context omitted.

Trust me there are plenty of us using cloud AI to actually ship stuff. We just aren't writing blog posts about it.

what did you ship?

I’ve used it for several improvements to our wysiwyg. I was able to refactor to a much more cohesive architecture that is making future work much easier/safer. We got a fzf like search algorithm for an autocomplete that includes marking which characters matched. It fixed some jank performance issues I was struggling to track down.

Fixed small several bugs last week without even open the editor. It wrote tests that confirmed the bugs, then fixed them. Then I manually confirmed as well.

It built a double entry event log. A single application event produces two logs, one for the person doing the action, one for the person being acted upon.

For a nasty problem we have, we had AI prototype 2 different solutions. Then we reveiewed them as a team. We threw those out but it was very helpful input into our decision making.

Re: I burned all my tokens researching how to save tokens

#193

Post by ncphillips »

6510 wrote: post #129
dominotw wrote: post #30

Earlier quoted context omitted.

what did you ship?

Thinking about it a better question [for me] would be what might have but didn't ship because of LLMs.

That’s an interesting question, in particular because I don’t think I have a single example

Re: I burned all my tokens researching how to save tokens

#194

Post by bkotrys »

_pdp_ wrote: post #187

The reason most people burn through their allocated limits is mostly because of sub-agents. Stop using them and you will find out that even the Pro tier is enough for daily 8-12h coding sessions. Don't use Fable though.

I don’t think subagents are the problem. Uncontrolled fan-out is. Separate contexts and independent analysis are extremely useful for larger tasks. In my experience, Pro is far too limited for sustained programming, although that probably depends a lot on the kind of engineering work you do.

Fable is great, but overkill for most tasks. Sonnet and Opus are already good enough for the execution work. I find Fable much more useful for planning, orchestration, advising, and judgment.

Re: I burned all my tokens researching how to save tokens

#195

Post by faeyanpiraat »

camgunz wrote: post #176
kolinko wrote: post #174

Earlier quoted context omitted.

Your fundamental math is wrong

no ur wrong

Well you both seem like well educated people, so can we bring in some thinking here and determine whether it is 2.5% or 10%?

How could this be verified?

I don't think I'm good enough at math and ai research to find a reliable way.

Re: I burned all my tokens researching how to save tokens

#196

Post by ncphillips »

dofm wrote: post #41
stavros wrote: post #33

Earlier quoted context omitted.

Whenever I've answered this question, the reply was always "this is shit", so I don't bother now. Bad faith questions just shouldn't be answered.

If you characterise questions as bad faith so you don’t have to answer, sure. Since this is my thread, and since I am an exasperated freelancer who has made most of his post-dot-com career from writing exactly the kind of quite small, simple, unambitious, often internal things for smaller customers that nobody would confuse with anything cool, who is trying to understand how AI is going to make his life any better wh…

I work at a 6 year old startup. Essentially every feature and bug fix we’ve shipped in the last year has had LLMs involved. Every project plan involved AI, either for writing or for codebase research. For code review we get AI to do the first pass.

Re: I burned all my tokens researching how to save tokens

#197

Post by Xalutiono »

dofm wrote: post #12

It sometimes feels to me that cloud AI providers have convinced people that cloud AI is worth it because of all the ways people have been able to use cloud AI to write blog posts about using cloud AI to make cloud AI more efficient for something that they haven't shipped yet and aren't really ready to talk about.

My companie spends a few million per month in token.

For me personally, the only reason why i don't use it 24/7 is cost.

Token costs, energy cost, usage limit due to token/subscription cost.

Re: I burned all my tokens researching how to save tokens

#198

Post by kittoes »

lenkite wrote: post #168
kittoes wrote: post #115

Earlier quoted context omitted.

Dying? Really? https://github.com/ByteTerrace/Puck/blob/main/src/Puck.Maths... Some of us aren't just slopping it up like @theo and all the other AI influencers are, as I developed nearly every single line of this by hand over the span of the past decade. Agents recently became capable of factchecking it and helped find bugs that my other unit testing never caught. Software engineering has never been more alive! http…

I am just confused on how you know that all of what has been generated is valid. When you write the code generator yourself, you have that level of confidence.

It's mathematically provable in most cases because the engine is deterministic from the ground up and I refused to ever compromise on that one detail. For example, nearly all of the VM and maths code functions have been validated for ALL possible 32-bit inputs and extensively fuzz tested in the 64-bit range. Even the more complicated sections of the code like the emulators have batteries of third party tests that ensure a certain level of compliance. The fact that you can play commerical ROMs to completion, including esoteric titles that require third party peripheral/MBC code, is quite solid proof too!

A form of recursion is also doing a lot of heavy lifting because the emulators are their own separate deterministic engines that are dynamically plumbed into the primary loop. This not only allows us to embed diegetic devices (as in, you can literally pick up a glowing brick in the world, hold it to your face, and play it), but also validates that the primary loop is coded accurately because the outer loop runs at X Hz while humble bricks must run at ~59.7275 Hz. Pacing inaccuracies in the main loop cause us to fail the brick suites that we test against. This concept of having features reinforce one another is applied as often as possible.

The obsession with determinism goes so far that both the DirectX and Vulkan backends are validated against each other for pixel level differences. If they differ more than the expected maximum then code is refactored until the drift is back within acceptable range.

Re: I burned all my tokens researching how to save tokens

#199

Post by Xalutiono »

camgunz wrote: post #54

TFA says "no hallucinations" but you can't fix hallucinations with rules or other models. I know I'm screaming into the void but whatever.

You can reduce hallucinations with rules and guardrailes for sure.

And Anthropic clearly shows that we can reduce hallucination in models. Progress is made which you can see in benchmarks.

And at the end of the day, a normal human being doesn't 'hallucinate' but 'missunderstand' things or just doesn't know what they are missing.

A LLM doesn't has to be perfect, it just has to be cheaper with the min. same quality as a human.

So why are you screaming into the void? Because LLM leave deterministic world? A lot of people don't even life in that world anyway.

Post reply on HN