Anyone else worried about the long term consequences of the influence of talking like this all day for the cognitive system of the user?
Caveman: Why use many token when few token do trick
151–160 of 396 posts
Re: Caveman: Why use many token when few token do trick
#152Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
[flagged]
Looking at the skill.md wouldn’t this actually increase token use since the model now needs to reformat its output?
Funny idea though. And I’d like to see a more matter-of-fact output from Claude.
Re: Caveman: Why use many token when few token do trick
#153Earlier quoted context omitted.
Can't you just know that the earth is the center of the world by... like... just looking at how the world works?
Actually you'd trivially disprove that claim if you're starting from mechanistic knowledge of how orbits work, like how we have mechanistic knowledge of how LLMs work.
You have empirical observations, like replicating a fixed set of inner layers to make it think longer, or that you seem to have encode and decode layers. But exactly why those layers are the way they are, how they come together for emergent behaviour... Do we have mechanistic knowledge of that?
Re: Caveman: Why use many token when few token do trick
#154Earlier quoted context omitted.
[flagged]
Let’s see, I think these pretty much map out a little chronology of the research: https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2406.06467 https://arxiv.org/abs/2404.15758 https://arxiv.org/abs/2512.12777 First that scratchpads matter, then why they matter, then that they don’t even need to be meaningful tokens, then a conceptual framework for the whole thing.
I dont’t see the relevance, the discussion is over whether boilerplate text that occurs intermittently in the output purely for the sake of linguistic correctness/sounding professional is of any benefit. Chain of thought doesn’t look like that to begin with, it’s a contiguous block of text.
Re: Caveman: Why use many token when few token do trick
#155Earlier quoted context omitted.
Can't you know that tokens are units of thinking just by... like... thinking about how models work?
> Can't you know that tokens are units of thinking just by... like... thinking about how models work? Seems reasonable, but this doesn't settle probably-empirical questions like: (a) to what degree is 'more' better?; (b) how important are filler words? (c) how important are words that signal connection, causality, influence, reasoning?
Right, there's probably something more subtle like "semantic density within tokens is how models think"
So it's probably true that the "Great question!---" type preambles are not helpful, but that there's definitely a lower bound on exactly how primitive of a caveman language we're pushing toward.
Re: Caveman: Why use many token when few token do trick
#156Earlier quoted context omitted.
Let’s see, I think these pretty much map out a little chronology of the research: https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2406.06467 https://arxiv.org/abs/2404.15758 https://arxiv.org/abs/2512.12777 First that scratchpads matter, then why they matter, then that they don’t even need to be meaningful tokens, then a conceptual framework for the whole thing.
I dont’t see the relevance, the discussion is over whether boilerplate text that occurs intermittently in the output purely for the sake of linguistic correctness/sounding professional is of any benefit. Chain of thought doesn’t look like that to begin with, it’s a contiguous block of text.
I don't see the relevance -- and casually dismiss years of researches without even trying to read those paper.
Re: Caveman: Why use many token when few token do trick
#157[flagged]
How is your offering different from local ollama?
Re: Caveman: Why use many token when few token do trick
#158Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
[flagged]
That "unproven claim" is actually a well-established concept called Chain of Thought (CoT). LLMs literally use intermediate tokens to "think" through problems step by step. They have to generate tokens to talk to themselves, debug, and plan. Forcing them to skip that process by cutting tokens, like making them talk in caveman speak, directly restricts their ability to reason.
Re: Caveman: Why use many token when few token do trick
#159Funny how people are so critical of this and yet fawn over TOON
Re: Caveman: Why use many token when few token do trick
#160Earlier quoted context omitted.
Let’s see, I think these pretty much map out a little chronology of the research: https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2406.06467 https://arxiv.org/abs/2404.15758 https://arxiv.org/abs/2512.12777 First that scratchpads matter, then why they matter, then that they don’t even need to be meaningful tokens, then a conceptual framework for the whole thing.
I dont’t see the relevance, the discussion is over whether boilerplate text that occurs intermittently in the output purely for the sake of linguistic correctness/sounding professional is of any benefit. Chain of thought doesn’t look like that to begin with, it’s a contiguous block of text.
That is not how CoT works. It is all in context. All influenced by context. This is a common and significant misunderstanding of autoregressive models and I see it on HN a lot.