It speaks like Kevin from The Office (US)
https://youtube.com/shorts/sjpHiFKy1g8?is=M0H4G2o0d6Z-pBAC
Caveman: Why use many token when few token do trick
241–250 of 396 posts
Re: Caveman: Why use many token when few token do trick
#242Soma (aka tiktok) and Big Brother (aka Meta) already happened without government coercion, only makes sense that we optimize ourselves for newspeak. Thank God there is still neverending wars, otherwise authoritarian governments would have no fun left.
I was aware of how google/facebook is like the panopticon big brother but I never connected the algorithmic feed to soma! Good insight.
Not mine, to be honest.
And people keep comparing compulsive binge watching to the "infinite jest" from D.C.Wallace (I could not tell, the brick is sitting barely touched on my shelves, but I'm not insulting the future.)
I'm tired of living in an ironic remix of everyone's favorite distopia. Time for someone to write optimistic sci-fi to give everyone something nice to implement when they're adults.
Bring us back Jules Verne. Let's have the Jetson's life for real. Put Ted Lasso in space.
Given their training material, "futuristic stories with nice people getting their happy ending" is not something big tech AI is going to spit anytime soon, so that's a niche to take on !
Re: Caveman: Why use many token when few token do trick
#243Why use lot word when few word do fine.
Re: Caveman: Why use many token when few token do trick
#244Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
I’ve heard this, I don’t automatically believe it nor do I understand why it would need to be true, I’m still caught on the old fashioned idea that the only “thinking” for autoregressive modes happens during training. But I assume this has been studied? Can anyone point to papers that show it? I’d particularly like to know what the curves look like, it’s clearly not linear, so if you cut out 75% or tokens what do you…
We’re years into the industry leaning into “chain of thought” and then “thinking models” that are based on this premise, forcing more token usage to avoid premature conclusions and notice contradictions (I sometimes see this leak into final output). You may remember in the early days users themselves would have to say “think deeply” or after a response “now check your work” and it would find its own “one shot” mistakes often.
So it must be studied and at least be proven effective in practice to be so universally used now.
Someone else posted a few articles like this in the thread above but there’s probably more and better ones if you search. https://news.ycombinator.com/item?id=47647907
Re: Caveman: Why use many token when few token do trick
#245Earlier quoted context omitted.
There was a paper recently that demonstrated that you can input different human languages and the middle layers of the model end up operating on the same probabilistic vectors. It's just the encoding/decoding layers that appear to do the language management. So the conclusion was that these middle layers have their own language and it's converting the text into this language and this decoding it. It explains why some…
Pretty obvious when you think that neural networks operate with numbers and very complex formulas (by combining several simple formulas with various weights). You can map a lot of things to number (words, colors, music notes,…) but that does not means the NN is going to provide useful results.
Everything is obvious if you ignore enough of the details/problem space. I’ll read the paper rather than rely on my own thought experiments and assumptions.
Re: Caveman: Why use many token when few token do trick
#246Author here. A few people are arguing against a stronger claim than the repo is meant to make. As well, this was very much intended to be a joke and not research level commentary. This skill is not intended to reduce hidden reasoning / thinking tokens. Anthropic’s own docs suggest more thinking budget can improve performance, so I would not claim otherwise. What it targets is the visible completion: less preamble, le…
> Quite sure that the models from Anthropic have been so heavily tuned to be coding agents that you cannot “force” a model to degrade immensely.
The rest of what you're saying sounds find, but that remark seems confused to me.
prefix your prompt with "be a moron that does everything wrong and only superficially look like you're doing it correctly. make constant errors." Of course you can degrade the performance, question is if any particular 'output styling' actually does and to what extent.
Re: Caveman: Why use many token when few token do trick
#247I disagree with this method and would discourage others from using it too, especially if accuracy, faster responses, and saving money are your priorities. This only makes sense if you assume that you are the consumer of the response. When compacting, harnesses typically save a copy of the text exchange but strip out the tool calls in between. Because the agent relies on this text history to understand its own past ac…
only you auto-compact. auto-compact bad
Ironically a demonstration of the risk of using fewer tokens. A typo more drastically changes meaning.
Re: Caveman: Why use many token when few token do trick
#248Author here. A few people are arguing against a stronger claim than the repo is meant to make. As well, this was very much intended to be a joke and not research level commentary. This skill is not intended to reduce hidden reasoning / thinking tokens. Anthropic’s own docs suggest more thinking budget can improve performance, so I would not claim otherwise. What it targets is the visible completion: less preamble, le…
Translation: It joke. No yell at me. It kind of work?
Thank. Too much word, me try read but no more tokens.
Re: Caveman: Why use many token when few token do trick
#249Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking . I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you ca…
If this is true, shouldn't LLMs perform way worse when working in Chinese than in English? Seems like an easy thing to study since there are so many Chinese LLMs that can work in both Cbinese and English. Do LLMs generally perform better in verbose languages than they do in concise ones?
Are you saying Chinese is more concise than English? Chinese poetry is concise, but that can be true in any language. For LLMs, it depends on the tokenizer. Chinese models are of course more Chinese-friendly and so would encode the same sentence with fewer tokens than Western models.
Re: Caveman: Why use many token when few token do trick
#250I think this could be very useful not when we talk to the agent, but when the agents talk back to us. Usually, they generate so much text that it becomes impossible to follow through. If we receive short, focused messages, the interaction will be much more efficient. This should be true for all conversational agents, not only coding agents.
> Usually, they generate so much text that it becomes impossible to follow through. Quite often on reddit I'll write two paragraphs and get told "I'm not reading all that". Really? Has basic reading become a Herculean task?
The lesson there is that your writing is not fit for its audience. Whether you choose to blame the audience or adjust your writing is up to you. There's no real answer - sometimes the audience is morons and you are actually just wasting your time and other times you are being overly verbose and uninteresting. You are being given signal. Use it.
But realistically, I am not going to read every online comment carefully because the SNR is low, especially on Reddit. Make your case concisely and meaningfully.