Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caveman: Why use many token when few token do trick

github.com

Post reply on HN
Print view View original

11–20 of 396 posts

Re: Caveman: Why use many token when few token do trick

#11

Post by baq »

TeMPOraL wrote: post #5

Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…

Do you know of evals with default Claude vs caveman Claude vs politician Claude solving the same tasks? Hypothesis is plausible, but I wouldn’t take it for granted

Re: Caveman: Why use many token when few token do trick

#13

Post by ryanschaefer »

Kinda ironic this description is so verbose.

> Use when user says "caveman mode", "talk like caveman", "use caveman", "less tokens", "be brief", or invokes /caveman

For the first part of this: couldn’t this just be a UserSubmitPrompt hook with regex against these?

See additionalContext in the json output of a script: https://code.claude.com/docs/en/hooks#structured-json-output

For the second, /caveman will always invoke the skill /caveman: https://code.claude.com/docs/en/skills

Re: Caveman: Why use many token when few token do trick

#15

Post by saidnooneever »

LOL it actually reads how humans reply the name is too clever :').

Not sure how effective it will be to dirve down costs, but honestly it will make my day not to have to read through entire essays about some trivial solution.

tldr; Claude skill, short output, ++good.

Re: Caveman: Why use many token when few token do trick

#16

Post by gozzoo »

I think this could be very useful not when we talk to the agent, but when the agents talk back to us. Usually, they generate so much text that it becomes impossible to follow through. If we receive short, focused messages, the interaction will be much more efficient. This should be true for all conversational agents, not only coding agents.

Re: Caveman: Why use many token when few token do trick

#19

Post by pixelpoet »

gozzoo wrote: post #16

I think this could be very useful not when we talk to the agent, but when the agents talk back to us. Usually, they generate so much text that it becomes impossible to follow through. If we receive short, focused messages, the interaction will be much more efficient. This should be true for all conversational agents, not only coding agents.

> Usually, they generate so much text that it becomes impossible to follow through.

Quite often on reddit I'll write two paragraphs and get told "I'm not reading all that".

Really? Has basic reading become a Herculean task?

Re: Caveman: Why use many token when few token do trick

#20

Post by p2detar »

gozzoo wrote: post #16

I think this could be very useful not when we talk to the agent, but when the agents talk back to us. Usually, they generate so much text that it becomes impossible to follow through. If we receive short, focused messages, the interaction will be much more efficient. This should be true for all conversational agents, not only coding agents.

That’s what it does as far as I get it. But less is not always better and I guess it’s also subjective to the promoter.
Post reply on HN