351–360 of 396 posts
Wouldn't this affect quality of output negatively? Thanks to chain of thought, actually having the LLM be explicit in its output allows it to have more quality.
Caveman only strips filler from what you see... the reasoning depth stays the same.
I found this visualisation pretty interesting - https://vectree.io/c/chain-of-thought-reasoning-how-llms-thi...
This is neat but my employer rates my performance based on token consumption; is there one that makes Claude needlessly verbose?
Is this a joke, or are you serious? Do you work for Nvidia?
2026 Boss: "Let's look at the AI tokens you used today."
The technology changes, but the micromanagement layer stays exactly the same.
Time is a circle, my friend. (=
Author here. A few people are arguing against a stronger claim than the repo is meant to make. As well, this was very much intended to be a joke and not research level commentary. This skill is not intended to reduce hidden reasoning / thinking tokens. Anthropic’s own docs suggest more thinking budget can improve performance, so I would not claim otherwise. What it targets is the visible completion: less preamble, le…
> There is actual research suggesting concise prompting can reduce response length substantially without always wrecking quality, Anecdote: i discussed that with an LLM once and it explained to me that LLMs tend to respond to terse questions with terse answers because that's what humans (i.e. their training data) tend to do. Similarly, it explained to me that polite requests tend to lead to LLM responses with _more_…
Do you have any idea how dumb this sounds?
Cute idea, but you're never gonna blow your token budget on output. Input tokens are the bottleneck, because the agent's ingesting swathes of skills, directory trees, code files, tool outputs, etc. The output is generally a few hundred lines of code and a bit of natural language explanation.
In single-turn use, yeah, but across dozens of turns there's probably value in optimizing the output. Btw your point lands just as well without "Cute idea, but" https://odap.knrdd.com/patterns/condescending-reveal
Earlier quoted context omitted.
Hmm... this sounds a lot like the old RISC vs CISC argument all over again. RISC won because simplicity scales better and you can always define complex instructions in terms of simple ones. So while I would relish experiencing the timeline in which our computerized chums bootstrap into sentience through the judicious application of carefully selected and highly nuanced words, it's playing out the other way: LLMs doin…
RISC good. CISC bad. But CISC tribe sneaky — hide RISC inside. Look CISC outside, think RISC inside. Trick work long time. Then ARM come. ARM very RISC. ARM go in phone. ARM go in tablet. ARM go everywhere. Apple make ARM chip, beat x86 with big club. Many impressed. Now ARM take server too. x86 tribe scared. RISC-V new baby RISC. Free for all. Many tribe use. Watch this one. RISC win brain fight. x86 survive by lyin…
Earlier quoted context omitted.
Yea you don't know anything about LLM architectures. They often change with each model release. You also aren't aware that there's more to it than "LLM architecture". And you're rather confident despite your lack of knowledge. You're like the old LLMs before ChatGPT was released that were kinda neat, but usually wrong and overconfident about it.
It's still attention and next-token-prediction and nothing else. The only new innovation is MoE, something that's used to optimize local models and not for the "SOTA" cloud offerings you're so fond of.
Earlier quoted context omitted.
What do you mean? It is grounded on the text it is fed, the reason it said that was that humans have said that or something similar to it, not because it analyzed a lot of LLM information and thought up that answer itself. LLM can "think" but that requires a lot of tokens to do, all quick answers are just human answers or answers it was fed with some basic pattern matching / interpolation.
There's nothing "basic" about the several months of training used to create a frontier model.
Earlier quoted context omitted.
> There is actual research suggesting concise prompting can reduce response length substantially without always wrecking quality, Anecdote: i discussed that with an LLM once and it explained to me that LLMs tend to respond to terse questions with terse answers because that's what humans (i.e. their training data) tend to do. Similarly, it explained to me that polite requests tend to lead to LLM responses with _more_…
this continual down-voting is not a personal thing for sure. perhaps there are crawlers that pretend to be more humane, or fully automated llm commenters which also randomly downvote.
If this really works there would seem to be a lot of alpha in running the expensive model in something like caveman mode, and then "decompressing" into normal mode with a cheap model. I don't think it would be fundamentally very surprising if something like this works, it seems like the natural extension to tokenisation. It also seems like the natural path towards "neuralese" where tokens no longer need to correspond…
Earlier quoted context omitted.
It is text prediction. But to predict text, other things follow that need to be calculated. If you can step back just a minute, i can provide a very simple but adjacent idea that might help to intuit the complexity of “ text prediction “ . I have a list of numbers, 0 to9, and the + , = operators. I will train my model on this dataset, except the model won’t get the list, they will get a bunch of addition problems. A…
>internalize the concepts. This gives the impression that it is doing something more than pattern matching. I think this kind of communication where some human attribute is used to name some concept in the LLM domain is causing a lot of damage, and ends up inadvertently blowing up the hype for the AI marketing...
I think what's causing a lot of damage is not attributing more of human attributes (though carefully). It's not the LLM marketing you have to worry about - that's just noise. All marketing is malicious lies and abusive bullshit, AI marketing is no different.
Care about engineering - designing and securing systems. There, the refusal to anthropomorphise LLMs is doing a lot of damage and wasted efforts, with good chunk of the industry believing in "lethal trifecta" as if it were the holy Trinity, and convinced it's something that can be solved without losing all that makes LLMs useful in the first place. A little bit of anthropomorphising LLMs, squinting your eyes and seeing them as little people on a chip, will immediately tell you these "bugs" and "vulnerabilities" are just inseparable facets of the features we care about, fundamental to general-purpose tools, and they can be mitigated and worked around (at a cost), but not solved, not any more you can solve "social engineering" or better code your employees so they're impervious to coercion or bribery, or being prompt-injected by a phone call from their loved one.