Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caveman: Why use many token when few token do trick

github.com

Post reply on HN
Print view View original

331–340 of 396 posts

Re: Caveman: Why use many token when few token do trick

#331

Post by Jensson »

codethief wrote: post #320
sumeno wrote: post #307

Earlier quoted context omitted.

LLMs don't understand what they are doing, they can't explain it to you, it's just creating a reasonable sounding response

But that response is grounded in the training data they've seen, so it's not entirely unreasonable to think their answer might provide actual insights, not just statistical parroting.

What do you mean? It is grounded on the text it is fed, the reason it said that was that humans have said that or something similar to it, not because it analyzed a lot of LLM information and thought up that answer itself.

LLM can "think" but that requires a lot of tokens to do, all quick answers are just human answers or answers it was fed with some basic pattern matching / interpolation.

Re: Caveman: Why use many token when few token do trick

#334

Post by avadodin »

dTal wrote: post #202
TeMPOraL wrote: post #5

Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…

Yeah but not all tokens are created equal. Some tokens are hard to predict and thus encode useful information; some are highly predictable and therefore don't. Spending an entire forward pass through the token-generation machine just to generate a very low-entropy token like "is" is wasteful . The LLM doesn't get to "remember" that thinking, it just gets to see a trivial grammar-filling token that a very dumb LLM cou…

> stenographically hiding steganographically*

Re: Caveman: Why use many token when few token do trick

#338

Post by johnfn »

konaraddi wrote: post #277
nayroclade wrote: post #33

Cute idea, but you're never gonna blow your token budget on output. Input tokens are the bottleneck, because the agent's ingesting swathes of skills, directory trees, code files, tool outputs, etc. The output is generally a few hundred lines of code and a bit of natural language explanation.

In single-turn use, yeah, but across dozens of turns there's probably value in optimizing the output. Btw your point lands just as well without "Cute idea, but" https://odap.knrdd.com/patterns/condescending-reveal

Pretty neat site you've got there. You should submit it to Show HN. I had fun clicking around - it's like TVTropes, except the examples make me angry, lol.

It would be pretty fun to train an LLM on this site and then have it flag my comments before I get downvoted, haha.

Re: Caveman: Why use many token when few token do trick

#339

Post by 7granddad »

trueno wrote: post #253
Chance-Device wrote: post #218

Earlier quoted context omitted.

Sounds reasonable to me. I think this thread is just the way online discourse tends to go. Actually it’s probably better than average, but still sometimes disappointing.

i played with this a bit the other night and ironically i think everyone should give it a shot as an alternative mode they might sometimes switch into. but not to save tokens, but instead to.. see things in a different light. its kind of great for the "eli5", not because it's any more right or wrong, but sometimes presenting it in caveman presents something to me in a way that's almost like... really clear and simple…

Interesting point! Based on what you said, in a way caveman does save your human brain tokens. Grammar rules evolve in a particular environment to reduce ambiguities and I think we are all familiar enough with caveman for it to make sense to all of us as a common. For example, word order matters for semantics in modern english so "The dog bit the grandma" and "Dog bit grandma" mean the same. Coming from languages where cases matter for semantics (like German), word order alone does not resolve ambiguity. Articles exist in English due to its Germanic roots

Re: Caveman: Why use many token when few token do trick

#340

Post by dr_kiszonka »

I appreciate the effort you put into addressing the feedback and updating the readme. I think the web design of your page and visual distractions in the readme go against the caveman's no-fluff spirit and may not appeal to the folks that would otherwise be into your software. I like the software.
Post reply on HN