Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caveman: Why use many token when few token do trick

github.com

Post reply on HN
Print view View original

51–60 of 396 posts

Re: Caveman: Why use many token when few token do trick

#51

Post by pennaMan »

DrewADesign wrote: post #36
avaer wrote: post #28

Earlier quoted context omitted.

That was my first thought too -- instead of talk like a caveman you could turn off reasoning, with probably better results. Additionally, LLMs do not actually operate in text; much of the thinking happens in a much higher dimensional space that just happens to be decoded as text. So unless the LLM was trained otherwise, making it talk like a caveman is more than just theoretically turning it into a caveman.

> much of the thinking happens in a much higher dimensional space that just happens to be decoded as text. What do you mean by that? It’s literally text prediction, isn’t it?

>It’s literally text prediction, isn’t it?

you are discovering that the favorite luddite argument is bullshit

Re: Caveman: Why use many token when few token do trick

#52

Post by jaccola »

staminade wrote: post #40
Rexxar wrote: post #22

Earlier quoted context omitted.

> Someone didn't get the memo that for LLMs, tokens are units of thinking. Where do you get this memo ? Seems completely wrong to me. More computation does not translate to more "thinking" if you compute the wrong things (ie things that contribute significantly to the final sentence meaning).

That’s why you need filler words that contribute little to the sentence meaning but give it a chance to compute/think. This is part of why humans do the same when speaking.

Do you have any evidence at all of this? I know how LLMs are trained and this makes no sense to me. Otherwise you'd just put filler words in every input

e.g. instead of: "The square root of 256 is" you'd enter "errr The er square um root errr of 256 errr is" and it would miraculously get better? The model can't differentiate between words you entered and words it generated its self...

Re: Caveman: Why use many token when few token do trick

#53

Post by jaccola »

teekert wrote: post #10

Idk I try talk like cavemen to claude. Claude seems answer less good. We have more misunderstandings. Feel like sometimes need more words in total to explain previous instructions. Also less context is more damage if typo. Who agrees? Could be just feeling I have. I often ad fluff. Feels like better result from LLM. Me think LLM also get less thinking and less info from own previous replies if talk like caveman.

Yes because in most contexts it has seen "caveman" talk the conversations haven't been about rigorously explained maths/science/computing/etc... so it is less likely to predict that output.

Re: Caveman: Why use many token when few token do trick

#54

Post by samus »

andai wrote: post #3

No articles, no pleasantries, and no hedging. He has combined the best of Slavic and Germanic culture into one :)

Both Slavic languages and German have complex declination systems for nouns, verbs, and adjectives. Which is unlike stereotypical caveman speech.

Re: Caveman: Why use many token when few token do trick

#55

Post by DrewADesign »

pennaMan wrote: post #51
DrewADesign wrote: post #36

Earlier quoted context omitted.

> much of the thinking happens in a much higher dimensional space that just happens to be decoded as text. What do you mean by that? It’s literally text prediction, isn’t it?

>It’s literally text prediction, isn’t it? you are discovering that the favorite luddite argument is bullshit

Feel free to elucidate if you want to add anything to this thread other than vibes.

Re: Caveman: Why use many token when few token do trick

#56

Post by electroglyph »

DrewADesign wrote: post #55
pennaMan wrote: post #51

Earlier quoted context omitted.

>It’s literally text prediction, isn’t it? you are discovering that the favorite luddite argument is bullshit

Feel free to elucidate if you want to add anything to this thread other than vibes.

after you go from from millions of params to billions+ models start to get weird (depending on training) just look at any number of interpretability research papers. Anthropic has some good ones.

Re: Caveman: Why use many token when few token do trick

#57

Post by throw83849494 »

avaer wrote: post #28
TeMPOraL wrote: post #5

Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…

That was my first thought too -- instead of talk like a caveman you could turn off reasoning, with probably better results. Additionally, LLMs do not actually operate in text; much of the thinking happens in a much higher dimensional space that just happens to be decoded as text. So unless the LLM was trained otherwise, making it talk like a caveman is more than just theoretically turning it into a caveman.

You obviously do not speak other languages. Other cultures have different constrains and different grammar.

For example thinking in modern US English generates many thoughts, to keep correct speak at right cultural context (there is only one correct way to say People Of Color, and it changes every year, any typo makes it horribly wrong).

Some languages are far more expressive and specialized in logical conditions, conditionals, recursion and reasoning. Like eskimos have 100 words for snow, but for boolean algebra.

It is well proven that thinking in Chinese needs far less tokens!

With this caveman mod you strip out most of cultural complexities of anglosphere, make it easier for foreigners and far simpler to digest.

Re: Caveman: Why use many token when few token do trick

#58

Post by afro88 »

TeMPOraL wrote: post #5

Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…

IIUC this doesn't make the LLM think in caveman (thinking tokens). It just makes the final output show in caveman.

Re: Caveman: Why use many token when few token do trick

#59

Post by vova_hn2 »

TeMPOraL wrote: post #5

Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…

Yeah, I don't think that "I'd be happy to help you with that" or "Sure, let me take a look at that for you" carries much useful signal that can be used for the next tokens.
Post reply on HN