Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
blog.jetbrains.com
Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
1–10 of 44 posts
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#2As the test shows the real gain is somewhat marginal to most people, but still some ppl claim a wide skills portfolio is required to move faster or save tokens. Mixed feelings, I personally use superpowers as my basic setup - and only them.
What are your thoughts on it?
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#3Well 8%-10% saving without measured quality degradation is not nothing, specially now that newer models seem to use more tokens than previous ones.
Also it was tested on reasoning low, i'd have liked to have them tested on higher reasoning levels.
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#4Yes all the caveman spoke English like its freaking Flinstone. Like wtf is the caveman dilect, just english words with some randomly skipped? Why not try with actual other human languages and see if there is a token benefit in savings
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#5If you just want a smaller vocabulary, use French? If your goal is to communicate to an LLM, maybe saving tokens isn't the all in win, unless you like reading assert gronkHitThing(true)
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#6As the test shows the real gain is somewhat marginal to most people, but still some ppl claim a wide skills portfolio is required to move faster or save tokens. Mixed feelings, I personally use superpowers as my basic setup - and only them. What are your thoughts on it?
I use my local GPU. Yes, I realize it's not exactly a superpower, but it's technology that's pretty darn close to one.
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#7Well 8%-10% saving without measured quality degradation is not nothing, specially now that newer models seem to use more tokens than previous ones. Also it was tested on reasoning low, i'd have liked to have them tested on higher reasoning levels.
Ask, what is the function of grammar in language? I would argue two major functions. Forcing an encoder to marshal ideas into a coherent structure. Preserving the integrity of that structure when communicated and decoded.
Grammar sounds off? Was the message correctly understood; is there enough left to apply error correction and regenerate the intended message?
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#8I suspect this means that there's just a ~10% inefficiency in token to information mapping.
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#9Yes all the caveman spoke English like its freaking Flinstone. Like wtf is the caveman dilect, just english words with some randomly skipped? Why not try with actual other human languages and see if there is a token benefit in savings
Sounds like some good research. You should pursue it.
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#10gpt 4 moto razr wen?