Anyone else worried about the long term consequences of the influence of talking like this all day for the cognitive system of the user ?
I think good, less thinking for you, more thinking you will do
Caveman: Why use many token when few token do trick
171–180 of 396 posts
Re: Caveman: Why use many token when few token do trick
#172Earlier quoted context omitted.
Let’s see, I think these pretty much map out a little chronology of the research: https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2406.06467 https://arxiv.org/abs/2404.15758 https://arxiv.org/abs/2512.12777 First that scratchpads matter, then why they matter, then that they don’t even need to be meaningful tokens, then a conceptual framework for the whole thing.
I dont’t see the relevance, the discussion is over whether boilerplate text that occurs intermittently in the output purely for the sake of linguistic correctness/sounding professional is of any benefit. Chain of thought doesn’t look like that to begin with, it’s a contiguous block of text.
To boil it down: chain of thought isn’t really chain of thought, it’s just more token generation output to the context. The tokens are participating in computations in subsequent forward passes that are doing things we don’t see or even understand. More LLM generated context matters.
Re: Caveman: Why use many token when few token do trick
#173By the way why don't these LLM interfaces come with a pause button?
And a "prune here" button. It often happens that the interesting information is in the first paragraph or so, and the remainder is all just the LLM not knowing when to stop. This is super annoying as a conversation then ends up being 90% noise.
Pruning an assistant's response like that would break prompt caching.
Prompt caching is probably the single most important thing that people building harnesses think about and yet it's mind share in end users is virtually zero. If you had to think of all the weirdest, most seemingly baffling design decisions in an AI product, the answer to "why" is probably "to not break prompt caching".
Re: Caveman: Why use many token when few token do trick
#174Earlier quoted context omitted.
There was a paper recently that demonstrated that you can input different human languages and the middle layers of the model end up operating on the same probabilistic vectors. It's just the encoding/decoding layers that appear to do the language management. So the conclusion was that these middle layers have their own language and it's converting the text into this language and this decoding it. It explains why some…
Ok — that sounds more like a theory rather than an open-and-shut causal explanation, but I’ll read the paper.
You’re a literature cycle behind. ‘Middle-layer shared representations exist’ is the observed phenomenon; ‘why exactly they form’ is the theory.
You are also confusing ‘mechanistic explanation still incomplete’ with ‘empirical phenomenon unestablished.’ Those are not the same thing.
PS. Em dash? So you are some LLM bot trying to bait mine HN for reasoning traces? :D
Re: Caveman: Why use many token when few token do trick
#175While really useful now, I'm afraid that in the long run it might accelerate the language atrophy that is already happening. I still remember that people used to enter full questions in Google and write SMS with capital letters, commas and periods.
> I still remember that people used to enter full questions in Google I think that, in the early days of internet search, entering full questions actually produced worse results than just a bunch of keywords or short phrases. So it was a sign of a "noob", rather than a mark of sophistication and literacy.
“Sophistication and literacy” are orthogonal to the peculiarities of a black box search engine.
Those literate sophisticates would still be noobs at getting something useful from Google.
Re: Caveman: Why use many token when few token do trick
#176If this really works there would seem to be a lot of alpha in running the expensive model in something like caveman mode, and then "decompressing" into normal mode with a cheap model. I don't think it would be fundamentally very surprising if something like this works, it seems like the natural extension to tokenisation. It also seems like the natural path towards "neuralese" where tokens no longer need to correspond…
But it can't, we see models get larger and larger and larger models perform better. made such huge improvements, because it makes more text for the language model to process. Cavemanising (lossy compression) the output does it to the input as well.
Re: Caveman: Why use many token when few token do trick
#177Does this actually result in less compute, or is it adding an additional “translate into caveman” step to the normal output?
Re: Caveman: Why use many token when few token do trick
#178Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
A fundamental (but sadly common) error behind “tokens are units of thinking” is antropomorphising the model as a thinking being. That’s a pretty wild claim that requires a lot of proof, and possibly solving the hard problem, before it can be taken seriously. There’s a less magical model of how LLMs work: they are essentially fancy autocomplete engines. Most of us probably have an intuition that the more you give an a…
IMO "thinking" here means "computation", like running matrix multiplications. Another view could be: "thinking" means "producing tokens". This doesn't require any proof because it's literally what the models do.
As I understand it, the claim is: more tokens = more computation = more "thinking" => answer probably better.
Re: Caveman: Why use many token when few token do trick
#179Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
CoT token are usually controled via 'extended thinking' or 'adapted thinking'. CoT tokens are usually not affected by the system prompt. There is an effort parameter, though, which states to have an effect on accuracy for over all token consumption.
https://platform.claude.com/docs/en/build-with-claude/extend...
Re: Caveman: Why use many token when few token do trick
#180Earlier quoted context omitted.
[flagged]
the fact that more tokens = more smart should be expected given cot / thinking / other techniques that increase the model accuracy by using more tokens. Did you test that ""caveman mode"" has similar performance to the ""normal"" model?
That is part of it. They are also trained to think in very well mapped areas of their model. All the RHLF, etc. tuned on their CoT and user feedback of responses.