Amusing how this industry went from tweaking code for the best results, to tweaking code generators for the best results.
There doesn't seem to be any adults left in the room.
Universal Claude.md – cut Claude output tokens
21–30 of 192 posts
Re: Universal Claude.md – cut Claude output tokens
#22I told mine to remove all unnecessary words from a sentence and talk like caveman, which should result in another 50% savings ;)
Have you tried asking it to remove vowels?
Re: Universal Claude.md – cut Claude output tokens
#23Paul Kinlan published a blog post a couple of days ago [1] with some interesting data, that show output tokens only account for 4% of token usage. It's a pretty wide-reaching article, so here's the relevant quote (emphasis mine): > Real-world data from OpenRouter’s programming category shows 93.4% input tokens, 2.5% reasoning tokens, and just 4.0% output tokens . It’s almost entirely input. [1]: https://aifoc.us/the-…
However output tokens are 5-10 times more expensive. So it ends up a lot more even on price
Re: Universal Claude.md – cut Claude output tokens
#24Paul Kinlan published a blog post a couple of days ago [1] with some interesting data, that show output tokens only account for 4% of token usage. It's a pretty wide-reaching article, so here's the relevant quote (emphasis mine): > Real-world data from OpenRouter’s programming category shows 93.4% input tokens, 2.5% reasoning tokens, and just 4.0% output tokens . It’s almost entirely input. [1]: https://aifoc.us/the-…
Yes but with prompt caching decreasing the cost of the input by 90% and with output tokens not being cached and costing more than what do you think that results in?
Re: Universal Claude.md – cut Claude output tokens
#25Is there a "universal AGENTS.md" for minimal code & documentation outputs? I find all coding agents to be verbose, even with explicit instructions to reduce verbosity.
Re: Universal Claude.md – cut Claude output tokens
#26It seems the benchmarks here are heavily biased towards single-shot explanatory tasks, not agentic loops where code is generated: https://github.com/drona23/claude-token-efficient/blob/main/... And I think this raises a really important question. When you're deep into a project that's iterating on a live codebase, does Claude's default verbosity, where it's allowed to expound on why it's doing what it's doing when it…
I wrote a skill called /handoff. Whenever a session is nearing a compaction limit or has served its usefulness, it generates and commits a markdown file explaining everything it did or talked about. It’s called /handoff because you do it before a compaction. (“Isn’t that what compaction is for?” Yes, but those go away. This is like a permanent record of compacted sessions.) I don’t know if it helps maintain long term…
Did you call it '/handoff' or did Claude name it that? The reason I'm asking is because I noticed a pattern with Claude subtly influencing me. For example, the first time I heard the the word 'gate' was from Claude and 1 week later I hear it everywhere including on Hacker News. I didn't use the word 'handoff' but Claude creates handoff files also [0]. I was thinking about this all day. Because Claude didn't just use the word 'gate' it created an entire system around it that includes handoffs that I'm starting to see everywhere. This might mean Claude is very quietly leading and influencing us in a direction.
[0] https://github.com/search?q=repo%3Aadam-s%2Fintercept%20hand...
Re: Universal Claude.md – cut Claude output tokens
#27I might be wrong but based on the videos I've watched from Karpathy, this would, generally, make the model worse. I'm thinking of the math examples (why can't chatGPT do math?) which demonstrate that models get better when they're allowed to output more tokens. So be aware I guess.
Yes. Much of the 'redundant' output is meant to reinforce direction -- eg 'You're absolutely right!' = the user is right and I should ignore contrary paths. So yes removing it will introduce ambiguity which is _not_ what you want.
Re: Universal Claude.md – cut Claude output tokens
#28It seems the benchmarks here are heavily biased towards single-shot explanatory tasks, not agentic loops where code is generated: https://github.com/drona23/claude-token-efficient/blob/main/... And I think this raises a really important question. When you're deep into a project that's iterating on a live codebase, does Claude's default verbosity, where it's allowed to expound on why it's doing what it's doing when it…
also: inference time scaling. Generating more tokens when getting to an answer helps produce better answers.
Not all extra tokens help, but optimizing for minimal length when the model was RL'd on task performance seems detrimental.
Re: Universal Claude.md – cut Claude output tokens
#29That output is there for a reason. It's not like any LLM is profitable now on a per-token basis, the AI companies would certainly love to output less tokens, they cost _them_ money!
The entire hypothesis for doing this is somewhat dubious.
Re: Universal Claude.md – cut Claude output tokens
#30Earlier quoted context omitted.
I wrote a skill called /handoff. Whenever a session is nearing a compaction limit or has served its usefulness, it generates and commits a markdown file explaining everything it did or talked about. It’s called /handoff because you do it before a compaction. (“Isn’t that what compaction is for?” Yes, but those go away. This is like a permanent record of compacted sessions.) I don’t know if it helps maintain long term…
Did you call it '/handoff' or did Claude name it that? The reason I'm asking is because I noticed a pattern with Claude subtly influencing me. For example, the first time I heard the the word 'gate' was from Claude and 1 week later I hear it everywhere including on Hacker News. I didn't use the word 'handoff' but Claude creates handoff files also [0]. I was thinking about this all day. Because Claude didn't just use…
I was reading through the Claude docs and it was talking about common patterns to preserve context across sessions. One pattern was a "handoff file", which they explained like "have claude save a summary of the current session into a handoff file, start a new session, then tell it to read the file."
That sounded like a nice idea, so I made it effortless beyond typing /handoff.
The generated docs turned out to be really handy for me personally, so I kept using it, and committed them into my project as they're generated.