the real interesting question would be if it then does its language-based reasoning also in short form and if so if quality is impacted.
Caveman: Why use many token when few token do trick
271–280 of 396 posts
Re: Caveman: Why use many token when few token do trick
#272There’s a lot of debate about whether this reduces model accuracy, but this is basically Chinese grammar and Chinese vibe coding seems to work fine while (supposedly) using 30-40% less tokens
It's like googling... if you have skillz/experience you can google almost anything with 3-4 words...
Re: Caveman: Why use many token when few token do trick
#273I've always figured that constraining an LLM to speak in any way other than the default way it wants to speak, reduces its intelligence / reasoning capacity, as at least some of its final layers can be used (on a per-token basis) either to reason about what to say, or about how to say it, but not both at once. (And it's for a similar reason, I think, that deliberative models like rewriting your question in their own…
[deleted]
Re: Caveman: Why use many token when few token do trick
#274This is neat but my employer rates my performance based on token consumption; is there one that makes Claude needlessly verbose?
After every loop, instruct it to ELI5 what it did into `/tmp`.
Re: Caveman: Why use many token when few token do trick
#275So you are telling me I prompted llms the right way all along
Re: Caveman: Why use many token when few token do trick
#276Earlier quoted context omitted.
If this is true, shouldn't LLMs perform way worse when working in Chinese than in English? Seems like an easy thing to study since there are so many Chinese LLMs that can work in both Cbinese and English. Do LLMs generally perform better in verbose languages than they do in concise ones?
Are you saying Chinese is more concise than English? Chinese poetry is concise, but that can be true in any language. For LLMs, it depends on the tokenizer. Chinese models are of course more Chinese-friendly and so would encode the same sentence with fewer tokens than Western models.
> Are you saying Chinese is more concise than English?
Yeah, definitely. It lacks case and verb conjugations, plus whole classes of filler words, and words themselves are on average substantially shorter. If you listen to or read a hyper-literal transliteration of Chinese speech into English (you can find fun videos of this on Chinese social media), it even resembles "caveman speech" for those reasons.
If you look at translated texts and compare the English versions to the Chinese ones, the Chinese versions are substantially shorter. Same if you compare localization strings in your favorite open-source project.
It's also part of why Chinese apps are so information-dense, and why localizing to other languages often requires reorganizing the layout itself— languages like English just aren't as information-dense, pixel for pixel.
The difference is especially profound for vernacular Chinese, which is why Chinese people often note that text which "has a machine translation flavor" is over-specified and gratuitously prolix.
Maybe some of this washes out in LLMs due to tokenization differences. But Chinese texts are typically shorter than English texts and it extends to prose as well as poetry.
But yeah this is standard stuff: Chinese is more concise and more contextual/ambiguous. More semantic work is allocated in interpretation than with English, less is allocated in the writing/speaking.
Do you speak Chinese and experience the differences between Chinese and English differently? I'm a native English speaker and only a beginner in Chinese but I've formed these views in discussion with Chinese people who know some English as well.
Re: Caveman: Why use many token when few token do trick
#277Cute idea, but you're never gonna blow your token budget on output. Input tokens are the bottleneck, because the agent's ingesting swathes of skills, directory trees, code files, tool outputs, etc. The output is generally a few hundred lines of code and a bit of natural language explanation.
In single-turn use, yeah, but across dozens of turns there's probably value in optimizing the output.
Btw your point lands just as well without "Cute idea, but" https://odap.knrdd.com/patterns/condescending-reveal
Re: Caveman: Why use many token when few token do trick
#278Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
[flagged]
"Don't be snarky."
Re: Caveman: Why use many token when few token do trick
#279Author here. A few people are arguing against a stronger claim than the repo is meant to make. As well, this was very much intended to be a joke and not research level commentary. This skill is not intended to reduce hidden reasoning / thinking tokens. Anthropic’s own docs suggest more thinking budget can improve performance, so I would not claim otherwise. What it targets is the visible completion: less preamble, le…
Sounds reasonable to me. I think this thread is just the way online discourse tends to go. Actually it’s probably better than average, but still sometimes disappointing.
It's much easier to talk about how something is deficient/untested than to do the testing yourself.
The same site that complains so much about replication crises in science too...
Re: Caveman: Why use many token when few token do trick
#280Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
[flagged]
I assume you're a human but wow this is the type of forum bot I could really get behind.
Take it a step further and do kind of like that xkcd where you try to post and it rewrites it like this and if you want the original version you have to write a justification that gets posted too.
Chef's kiss