Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caveman: Why use many token when few token do trick

github.com

Post reply on HN
Print view View original

261–270 of 396 posts

Re: Caveman: Why use many token when few token do trick

#261

Post by dataviz1000 »

JBrussee-2 wrote: post #208

Author here. A few people are arguing against a stronger claim than the repo is meant to make. As well, this was very much intended to be a joke and not research level commentary. This skill is not intended to reduce hidden reasoning / thinking tokens. Anthropic’s own docs suggest more thinking budget can improve performance, so I would not claim otherwise. What it targets is the visible completion: less preamble, le…

If you want to benchmark, consider this https://github.com/adam-s/testing-claude-agent

Re: Caveman: Why use many token when few token do trick

#262

Post by wvenable »

shomp wrote: post #187

everyone who thinks this is a costly or bad idea is looking past a very salient finding: code doesn't need much language. sure, other things might need lots of language, but code does not. code is already basically language, just a really weird one. we call them programming languages. they're not human languages. they're languages of the machine. condensing the human-language---machine-language interface, good. if go…

> They're not human languages. they're languages of the machine.

Disagree. Programming language for human to communicate with machine and human and human to communicate about machine. Programming language not native language of machine. Programming language for humans.

Otherwise make good point.

Re: Caveman: Why use many token when few token do trick

#263

Post by ben_w »

freehorse wrote: post #194
taneq wrote: post #168

Earlier quoted context omitted.

Think before talk better though

Think makes smart. But think right words makes smarter, not think more words. Smart is elucidate structure and relationships with right words.

think make smart, llm approximate "think" with context, llm not smart ever but sometimes less dumb with more word

Re: Caveman: Why use many token when few token do trick

#264

Post by ben_w »

gchamonlive wrote: post #153
estearum wrote: post #148

Earlier quoted context omitted.

Actually you'd trivially disprove that claim if you're starting from mechanistic knowledge of how orbits work, like how we have mechanistic knowledge of how LLMs work.

You have empirical observations, like replicating a fixed set of inner layers to make it think longer, or that you seem to have encode and decode layers. But exactly why those layers are the way they are, how they come together for emergent behaviour... Do we have mechanistic knowledge of that?

I think we've *only* got the mechanism, not the implications.

Compare with fluid dynamics; it's not hard to write down the Navier–Stokes equations, but there's a million dollars available to the first person who can prove or give a counter-example of the following statement:

  In three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smooth and globally defined, that solve the Navier–Stokes equations.
- https://en.wikipedia.org/wiki/Navier–Stokes_existence_and_sm...

Re: Caveman: Why use many token when few token do trick

#267

Post by kubb »

prodigycorp wrote: post #92
kubb wrote: post #32

Earlier quoted context omitted.

This is condescending and wrong at the same time (best combo). LLMs do stumble into long prediction chains that don’t lead the inference in any useful direction, wasting tokens and compute.

Are you sure about that? Chain of thought does not need to be semantically useful to improve LLM performance. https://arxiv.org/abs/2404.15758

If you're misusing LLMs to solve TC^0 problems, which is what the paper is about, then... you also don't need the slop lavine. You can just inject a bunch of filler tokens yourself.

Re: Caveman: Why use many token when few token do trick

#268

Post by svachalek »

derefr wrote: post #239

I've always figured that constraining an LLM to speak in any way other than the default way it wants to speak, reduces its intelligence / reasoning capacity, as at least some of its final layers can be used (on a per-token basis) either to reason about what to say, or about how to say it, but not both at once. (And it's for a similar reason, I think, that deliberative models like rewriting your question in their own…

I think this is on point, I've really started to think about LLMs in terms of attention budget more than tokens. There's only so many things they can do at once, which ones are most important to you?
Post reply on HN