Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Universal Claude.md – cut Claude output tokens

github.com

Post reply on HN
Print view View original

181–190 of 192 posts

Re: Universal Claude.md – cut Claude output tokens

#181

Post by gavinray »

joshstrange wrote: post #16

As with all of these cure-alls, I'm wary. Mostly I'm wary because I anticipate the developer will lose interest in very little time and also because it will just get subsumed into CC at some point if it actually works. It might take longer but changing my workflow every few days for the new thing that's going to reduce MCP usage, replace it, compress it, etc is way too disruptive. I'm generally happy with the base Cl…

  > "because it will just get subsumed into CC at some point if it actually works."
This is the sharp-bladed axe of reason I've used against all of these massive "prompt frameworks" and "superprompts".

Anthropic's survival depends on Claude Code performing as well as it can, by all metrics.

If the Very Smart People working on CC haven't integrated a feature or put text into the System Prompt, it's probably because it doesn't improve performance.

Put another way: The product is probably as optimized as it can get when it comes out the box, and I'm skeptical about claims otherwise without substantial proof.

Re: Universal Claude.md – cut Claude output tokens

#182

Post by sossov »

btown wrote: post #4

It seems the benchmarks here are heavily biased towards single-shot explanatory tasks, not agentic loops where code is generated: https://github.com/drona23/claude-token-efficient/blob/main/... And I think this raises a really important question. When you're deep into a project that's iterating on a live codebase, does Claude's default verbosity, where it's allowed to expound on why it's doing what it's doing when it…

[dead]

Re: Universal Claude.md – cut Claude output tokens

#183

Post by andai »

esperent wrote: post #22
andai wrote: post #9

I told mine to remove all unnecessary words from a sentence and talk like caveman, which should result in another 50% savings ;)

Have you tried asking it to remove vowels?

Not sure that would help due to how tokenization works, but I remember from the early GPT-4 days that LLMs have the ability to "compress" a message into an incomprehensible string of Unicode, which the LLM itself understands perfectly, and which is 5-10x shorter than the English text.

That was a big deal when the context size was 8K; now that tokens are cheap and context is huge, nobody seems to be investigating that anymore.

Re: Universal Claude.md – cut Claude output tokens

#185

Post by aqfamnzc »

johnfn wrote: post #157
rimliu wrote: post #106

Earlier quoted context omitted.

there are no reasoning LLMs.

This is an interesting denial of reality.

A "reasoning" LLM is just an LLM that's been instructed or trained to start every response with some text wrapped in or similar. The UI may show or obscure this part. Then when the model decides to give its "real" response, it has all that reasoning text in its context window, helping it generate a better answer.

Re: Universal Claude.md – cut Claude output tokens

#187

Post by kinlan »

verdverm wrote: post #90
monooso wrote: post #15

Paul Kinlan published a blog post a couple of days ago [1] with some interesting data, that show output tokens only account for 4% of token usage. It's a pretty wide-reaching article, so here's the relevant quote (emphasis mine): > Real-world data from OpenRouter’s programming category shows 93.4% input tokens, 2.5% reasoning tokens, and just 4.0% output tokens . It’s almost entirely input. [1]: https://aifoc.us/the-…

My own output token ratio is 2% (50% savings on the expensive tokens, I include thinking in this, which is often more). I have similar tone and output formatting system prompt content.

That's actually useful to know and it aligns with what I see (I wrote the cost post)

Re: Universal Claude.md – cut Claude output tokens

#188

Post by kinlan »

weird-eye-issue wrote: post #56
wongarsu wrote: post #23

Earlier quoted context omitted.

However output tokens are 5-10 times more expensive. So it ends up a lot more even on price

Even more than that in practice once you factor in prompt caching

I think we still skew back to an insanely high input token ratio when you consider agentic loops. For example, when I see the tools I use do a web fetch or a search or other tool use, it's an incredibly high number of new input tokens.

Re: Universal Claude.md – cut Claude output tokens

#189

Post by btown »

dataviz1000 wrote: post #108
btown wrote: post #4

It seems the benchmarks here are heavily biased towards single-shot explanatory tasks, not agentic loops where code is generated: https://github.com/drona23/claude-token-efficient/blob/main/... And I think this raises a really important question. When you're deep into a project that's iterating on a live codebase, does Claude's default verbosity, where it's allowed to expound on why it's doing what it's doing when it…

I made a test [0] which runs several different configurations against coding tasks from easy to hard. There is a test which it has to pass. Because of temperature, the number of tokens per one shot vary widely with all the different configurations include this one. However, across 30 tests, this does perform worse. [0] https://github.com/adam-s/testing-claude-agent

This is an amazing analysis! Thank you for running this :)

Re: Universal Claude.md – cut Claude output tokens

#190

Post by motoboi »

nearbuy wrote: post #174
motoboi wrote: post #172

Earlier quoted context omitted.

The reasoning part is not diferente from the part that goes in answer. It’s just that the model is trained to do some magical text generation with back and forth. But when it’s writing the answer part of it, each word is part of its context when generating the next. What that means is that the model does not compute then write, it generates text that guide the next generation in the general direction of the answer. I…

> The reasoning part is not diferente from the part that goes in answer. Exactly. And this instruction isn't telling it to skip the reasoning. That part is unaffected. The instruction is only for the user-visible output. By the time the reasoning models get to writing the output you see, they've already decided what they are going to say. The answer is based on whatever it decided while reasoning. It doesn't matter w…

Im sorry. You are thinking in terms of one time interactions. I’m thinking about the next step in the interaction.

To understand my point, think about a prompt to tell the model “here is a very difficult code problem, answer in a single word.”

It thinks a lot and answer. You send the next prompt. At this moment, you are completely in out of distribution territory.

Post reply on HN