Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Models Are Getting Dumber on Purpose

w4g1.dev

Post reply on HN
Print view View original

171–180 of 197 posts

Re: Models Are Getting Dumber on Purpose

#171

Post by mypalmike »

jiggawatts wrote: post #133
kennywinker wrote: post #128

Earlier quoted context omitted.

Those assumptions are just that - assumptions. "Local council in XYZ location" implies a bunch of things, and each one might be wrong for my specific circumstances. What better way to guide expectations than importing specific knowledge? I.e. if I import the english and catalan modules, then I probably want to localize my site in english and catalan. It would be trivial to have a pre-flight convo with an llm to guide…

See: https://en.wikipedia.org/wiki/Bitter_lesson Everyone assumes that carefully crafting a specific AI architecture with bits and pieces bolted together based on their human intuition is necessarily superior to simply using a bigger monolithic AI model. It turns out that the opposite is true, and has been demonstrated over and over again. The bitter lesson is this: You can simply ask a frontier model to do the thing…

“But I can customise EMACS endlessly, why would I use an actual IDE?”

There’s no need to bring religion into this.

Re: Models Are Getting Dumber on Purpose

#172

Post by chicken-stew »

jimmaswell wrote: post #58
kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese p…

Ah yes, so that’s why if I ask gemini for a simple low-resource trick to cast a float to its binary representation it is reminded of a work of shakespeare and suggests memcpy.

Or yesterday, when I asked it for the best low resource approach to select the matching string from a collection of strings for a shell interface it suggested an exhaustive string compare over the entire collection. Because it was thinking of how David Lynch would tackle it.

Re: Models Are Getting Dumber on Purpose

#173

Post by 5-0 »

kentonv wrote: post #99
margalabargala wrote: post #91

Earlier quoted context omitted.

I agree with everything you say except this: > You can make any modern LLM explain its reasoning You can make any modern LLM create a plausible, self-consistent explanation that looks like reasoning, but it's not "the reasoning it used to arrive at that answer".

Tangent: This is often true of humans as well. We often make a decision based on a gut feeling, and then backfill a logical reason supporting our feeling, without even realizing we're doing it -- rationalization.

Yes, humans in general. Scientists, perhaps less so.

Is it unreasonable to hold LLMs to a higher standard?

Re: Models Are Getting Dumber on Purpose

#174

Post by angry_octet »

efilife wrote: post #167
angry_octet wrote: post #153

Earlier quoted context omitted.

And we all know some people that rationalise poor choices and misbehavior, hide their mistakes, etc, to an unacceptable degree. Sometimes the individual knows they are rationalising but continues anyway, other times they seem incapable of seeing that. When you ask people who are rationalising poor behaviour about the scenario, but it is someone else doing it, they may arrive at a better answer. Can we use multiple LL…

isn't this happening already? There's the concept called "thinking" where the models talks with itself before giving you the final answer

It does multiple rounds of feedback which is called 'thinking' but whether that is critical thinking is unknown.

Re: Models Are Getting Dumber on Purpose

#175

Post by JV00 »

kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

“I know Kung Fu” - Neo (The Matrix, 1999)

Re: Models Are Getting Dumber on Purpose

#176

Post by dist-epoch »

markasoftware wrote: post #140
dist-epoch wrote: post #94

Earlier quoted context omitted.

This was beautifully shown by asking a model to explain how it added two numbers together (something like 45+21), and it told a plausible story, when in fact they showed it was some rotation on a helix living in some internal manifold. Like asking a human "how did you catch that fast ball coming at you?"

It could be that the rotation in the helix manifold whatever is a low level representation of the logical steps (carry the 2, add the next column,...) it's describing. The point stands that the explanation it generates doesn't necessarily in all cases reflect what it "actually did" but your counterexample doesn't hold.

There have been multiple studies on how LLMs add numbers.

They use the "Clock" algorithm, the "Pizza" algorithm, a few other ones.

> All networks we study implement the same simple neuron model in their first-layer MLPs: degree-1 sinusoidal fits in layer 1, with deeper layers combining into degree-2 sinusoidal interactions.

https://neurips.cc/virtual/2025/loc/san-diego/133808

https://arxiv.org/abs/2502.00873

When you ask them they don't mention these at all, they give you high-school math:

https://chatgpt.com/share/6a82afdd-872c-83eb-aad7-622d27f2dc...

> did you use the "cos" or "sin" function at all during this addition computation?

> No. There is no need for trigonometric functions like sin or cos. The computation only uses basic arithmetic and place-value reasoning. Of course, if someone were implementing arithmetic in a computer, it is theoretically possible to express addition using extremely complicated formulas involving sin and cos. But in the reasoning I described, no trigonometric functions were involved at all. I simply decomposed the numbers into hundreds and smaller parts and added them.

Re: Models Are Getting Dumber on Purpose

#177

Post by genxy »

znnajdla wrote: post #70
jimmaswell wrote: post #58

Earlier quoted context omitted.

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese p…

Thanks for putting this so well. The mathematical evidence for this “general intelligence underlying everything” is the “universal geometry of embeddings” paper. Fascinating read. Or as the ancient philosophers used to say, the one who knows God knows everything.

Is this the paper?

Harnessing the Universal Geometry of Embeddings https://arxiv.org/abs/2505.12540

Re: Models Are Getting Dumber on Purpose

#178

Post by Ohentis »

8bitbeep wrote: post #146
jiggawatts wrote: post #133

Earlier quoted context omitted.

See: https://en.wikipedia.org/wiki/Bitter_lesson Everyone assumes that carefully crafting a specific AI architecture with bits and pieces bolted together based on their human intuition is necessarily superior to simply using a bigger monolithic AI model. It turns out that the opposite is true, and has been demonstrated over and over again. The bitter lesson is this: You can simply ask a frontier model to do the thing…

> It's the classic "But I can customise EMACS endlessly, why would I use an actual IDE?" argument all over. That paragraph sets me off. I’ll take Vim and Emacs over VSCode and Eclipse any day.

Well vscode is also not an actual ide

Re: Models Are Getting Dumber on Purpose

#179

Post by iugtmkbdfil834 »

znnajdla wrote: post #59
kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

This is a fundamental misunderstanding of how LLMs work. You can’t really specialize a model. You specialize the harness. A well-trained general purpose LLM doesn’t need examples in its training data, it can write good code in a new language you invented yesterday with just a spec definition. And it will perform better than a small model trained on lots of examples of your invented language. The reason is because of…

Chomsky would like to have a word. Your statement is true only at a surface leve l( they have nouns, verbs and some limits), but it breaks down the moment you start to inspect it closely:

- sign language ( I am not being petty; you put all languages ) - it is almost nothing like the underlying structure of other spoken languages primarily because it does not carry its restrictions - English vs Polish example - word can carry grammar or not; word order can carry meaning or not

Those are just two examples, but both clearly show that little about human languages is actually the same. It is kinda like the history thing. It rhymes.

I am addressing this part as other posters noted issues with other parts.

Re: Models Are Getting Dumber on Purpose

#180

Post by kennywinker »

chicken-stew wrote: post #172
jimmaswell wrote: post #58

Earlier quoted context omitted.

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese p…

Ah yes, so that’s why if I ask gemini for a simple low-resource trick to cast a float to its binary representation it is reminded of a work of shakespeare and suggests memcpy. Or yesterday, when I asked it for the best low resource approach to select the matching string from a collection of strings for a shell interface it suggested an exhaustive string compare over the entire collection. Because it was thinking of h…

David Lynch would simply refuse to compare strings. Which is how we can tell LLMs haven’t been trained on enough David Lynch - they don’t simply refuse often enough.
Post reply on HN