Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I burned all my tokens researching how to save tokens

quesma.com

Post reply on HN
Print view View original

201–210 of 237 posts

Re: I burned all my tokens researching how to save tokens

#201

Post by bkotrys »

a_c wrote: post #191
bkotrys wrote: post #184

Earlier quoted context omitted.

The audit point is especially interesting. Is your Claude token-audit tool available anywhere?

I didn't update it for a while. It reads your local conversation history, correlate input/output token with usage (claude doesn't give exact breakdown), and classify if they are subagent/tooluse/normal message/whatever. Let me know if I can make it any way useful to you. https://github.com/kmcheung12/ccaudit

Thanks, this looks useful. The subagent and tool-use breakdown is exactly the kind of visibility I was looking for. I will give it a try.

Re: I burned all my tokens researching how to save tokens

#202

Post by cgio »

dominotw wrote: post #30
resonious wrote: post #20

Earlier quoted context omitted.

Trust me there are plenty of us using cloud AI to actually ship stuff. We just aren't writing blog posts about it.

what did you ship?

I did ship a quite efficient quantum simulator as a toy out of other work (and a block truncation certificate as a side-product). https://github.com/cgiogkarakis/qut

Re: I burned all my tokens researching how to save tokens

#204

Post by [deleted] »

faeyanpiraat wrote: post #195
camgunz wrote: post #176

Earlier quoted context omitted.

no ur wrong

Well you both seem like well educated people, so can we bring in some thinking here and determine whether it is 2.5% or 10%? How could this be verified? I don't think I'm good enough at math and ai research to find a reliable way.

[deleted]

Re: I burned all my tokens researching how to save tokens

#205

Post by [deleted] »

faeyanpiraat wrote: post #195
camgunz wrote: post #176

Earlier quoted context omitted.

no ur wrong

Well you both seem like well educated people, so can we bring in some thinking here and determine whether it is 2.5% or 10%? How could this be verified? I don't think I'm good enough at math and ai research to find a reliable way.

[deleted]

Re: I burned all my tokens researching how to save tokens

#206

Post by _pdp_ »

bkotrys wrote: post #194
_pdp_ wrote: post #187

The reason most people burn through their allocated limits is mostly because of sub-agents. Stop using them and you will find out that even the Pro tier is enough for daily 8-12h coding sessions. Don't use Fable though.

I don’t think subagents are the problem. Uncontrolled fan-out is. Separate contexts and independent analysis are extremely useful for larger tasks. In my experience, Pro is far too limited for sustained programming, although that probably depends a lot on the kind of engineering work you do. Fable is great, but overkill for most tasks. Sonnet and Opus are already good enough for the execution work. I find Fable much…

I am not sure. I think of myself as a pro user yet I am finding hard time maxing out Pro tier. It only happens with sub-agents without any material returns.

Re: I burned all my tokens researching how to save tokens

#207

Post by wccrawford »

_pdp_ wrote: post #187

The reason most people burn through their allocated limits is mostly because of sub-agents. Stop using them and you will find out that even the Pro tier is enough for daily 8-12h coding sessions. Don't use Fable though.

I think this is almost a tautology, to the point that I'm surprised more people don't realize it.

If you ask the AI to do more work, it uses more tokens. If you do more of the work instead, fewer tokens are used.

I've been looking at Matt Pocock's skills recently, and they're very useful. But they use tokens. You can do that work yourself without the token usage, and just have the AI do the coding, or you can tell the AI to do everything and pay the cost.

And you can scale it down even further, and reduce the AI to a glorified code completion tool, and use way fewer tokens. But it's not going to speed up your work nearly as much.

For now, finding the right balance is important. In the future, I think things will eventually shift to the point that we just hand most of it to the AI and let it go without worrying about tokens.

Re: I burned all my tokens researching how to save tokens

#208

Post by killix »

bkotrys wrote: post #123
iririririr wrote: post #84

echo "$OUT" | claude-mem-save so, does agents are ultra aware their output is used in a bash line or did bash now have some magic quote super power? I'd bet this only really work 2% of the time in real world.

At the shell level, the agent does not need to know about Bash. The contents of "$OUT" are passed to echo as one argument, then sent to claude-mem-save over stdin. Bash does not re-evaluate them as shell code. printf '%s\n' "$OUT" would be more robust than echo. Whether claude-mem-save can use arbitrary output is a separate question.

[flagged]

Re: I burned all my tokens researching how to save tokens

#209

Post by juvvel »

faeyanpiraat wrote: post #195
camgunz wrote: post #176

Earlier quoted context omitted.

no ur wrong

Well you both seem like well educated people, so can we bring in some thinking here and determine whether it is 2.5% or 10%? How could this be verified? I don't think I'm good enough at math and ai research to find a reliable way.

This reminds of sensitivity vs. specificity for medical tests. You essentially have a 2x2 matrix: Error vs. non-error on one axis, and detected vs. not detected on the other. If we assume that a reviewer LLM has 95% accuracy, meaning that it produces the correct result 95% of the time it (identifying errors as errors, and identifying non-errors as non-errors), you get the following: Of 100 cases, the output of 5 is wrong, and the output of 95 is correct. Of the 5 wrong outputs, the LLM correctly identifies 4.75 (95%) as wrong. But it will also create 4.75 false positives (5% of 95). Of the 95 correct ones, the LLM correctly identifies 90.25 (95%) as correct. Therefore it creates 0.25 false negatives. So the positive predictive value is low: When the reviewer LLM flags an error, it's actually only an error 50% of the time. On the other hand, the negative predictive value is high, so in ~99.7% of cases, there will be no error if the reviewer LLM didn't catch one. All of that is assuming that the actual accuracy of the reviewer LLM is truly a flat 95% and that the base error rate is truly only 5%. I think that in reality, LLMs perform better with a certain type of tasks / errors and 5% error rate is just an average.

Re: I burned all my tokens researching how to save tokens

#210

Post by dominotw »

Thanemate wrote: post #189
password4321 wrote: post #116

Earlier quoted context omitted.

Most of "What are you working on?" is AI assisted; not much is shipped. https://hn.algolia.com/?query=working%20author%3Adavid927&so...

Checking all the past "what are you working on?" threads just to realize that, somewhere in August 2024 stuff just exploded and the average thread goes reached ~1000 messages, from ~50 pre-August.

Tons of ppl got into CS in last few years worldwide.
Post reply on HN