Why don't we just use a highly specialized domain-specific language that describes the code we want generated? Natural language is too inexact and ambiguous.
You mean like… some sort of code?
Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
41–44 of 44 posts
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#42Why not Mandarin Chinese? Logographic is pretty compact. Also, this really should be more precise that it’s talking about neo-caveman. Legit caveman no speak English.
chinese is less token efficient than english
How so? Do you mean in practice when using mostly english trained models?
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#43Does anyone more familiar with the technology know if these AI models treat abugidas like Thai or abjads like Arabic differently when it comes to token usage?
Specifically abjads since as far as I am aware their difference from alphabetical and abugida scripts is that they infer the vowels instead of explicitly marking them.
Re: Does Speaking to Agents Like Cavemen Save 65% of Tokens? We Test
#44Earlier quoted context omitted.
chinese is less token efficient than english
How so? Do you mean in practice when using mostly english trained models?
Even in chinese trained models too. the amount of tokens it requires to communicate the same ideas is higher. it is just less efficient to tokenize the Chinese language because logographic languages are archaic, ridiculous, and inefficient compared to latin languages.