Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Models Are Getting Dumber on Purpose

w4g1.dev

Post reply on HN
Print view View original

131–140 of 197 posts

Re: Models Are Getting Dumber on Purpose

#131

Post by fennecbutt »

kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

I was hoping model architecture would migrate towards something like this, perhaps it still might. Sparse models seem like they're getting more popular.

I'd always thought we'd eventually hotload loras or MoE experts.

It would certainly be useful on the robotics/VLA side of things as well; more limited mobile hardware, download and load/unload new skills as needed.

Tbf I also don't really care what facts my models have baked in (for llms at least). I care most that the model understands general logic and then general knowledge of some level is secondary. Reason being is that everything is RAG'd in anyway.

Models spitting out well established facts is cute but I don't really ever want to rely on say "electronics knowledge" that exists in a tenuous and vague form in the model weights.

Humans write books (and datasheets) for a reason. Books are RAG.

Re: Models Are Getting Dumber on Purpose

#132

Post by fennecbutt »

jiggawatts wrote: post #123
jimmaswell wrote: post #58

Earlier quoted context omitted.

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese p…

People keep forgetting that programming is not just about knowing the target programming language, but also an enormous volume of tacit knowledge : - Understanding of protocols like HTTP. - HTML, JS, CSS, SVG, and everything "web". - Understanding of databases, SQL, etc. - Abstract code architecture patterns. - Understanding the users' requests in English. - Responding in English. - Command line tool usage (agents/ha…

Tbf the majority of harnesses for frontier models do not allow the agent to do this (gut instinct) and instead get it to search, and for good reason.

Re: Models Are Getting Dumber on Purpose

#133

Post by jiggawatts »

kennywinker wrote: post #128
jiggawatts wrote: post #123

Earlier quoted context omitted.

People keep forgetting that programming is not just about knowing the target programming language, but also an enormous volume of tacit knowledge : - Understanding of protocols like HTTP. - HTML, JS, CSS, SVG, and everything "web". - Understanding of databases, SQL, etc. - Abstract code architecture patterns. - Understanding the users' requests in English. - Responding in English. - Command line tool usage (agents/ha…

Those assumptions are just that - assumptions. "Local council in XYZ location" implies a bunch of things, and each one might be wrong for my specific circumstances. What better way to guide expectations than importing specific knowledge? I.e. if I import the english and catalan modules, then I probably want to localize my site in english and catalan. It would be trivial to have a pre-flight convo with an llm to guide…

See: https://en.wikipedia.org/wiki/Bitter_lesson

Everyone assumes that carefully crafting a specific AI architecture with bits and pieces bolted together based on their human intuition is necessarily superior to simply using a bigger monolithic AI model. It turns out that the opposite is true, and has been demonstrated over and over again.

The bitter lesson is this: You can simply ask a frontier model to do the things you suggested, in a few terse lines of English. Dump a few lines in AGENTS.md and you are good to go.

Your approach is to "fiddle with inadequate tools" for weeks or months until you can finally attain a pale imitation of what the frontier models can do effortlessly.

It's the classic "But I can customise EMACS endlessly, why would I use an actual IDE?" argument all over.

I get it. You don't feel ownership over someone else's AI. You don't feel involved, you don't feel like you have agency.

It's like LEGO or IKEA furniture: study after study has shown that people enjoy things more if they "put it together themselves", even if fundamentally the thing is worse and/or still essentially nothing more than plastic made in a factory.

Re: Models Are Getting Dumber on Purpose

#134

Post by Gander5739 »

COAGULOPATH wrote: post #85

This AI generated post (100% on Pangram) is pretty out of date. >On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions. SimpleQA hasn't been updated in a long time. Gemini 2.5 Pro is a sixteen-month-old model, not "the best recall money can buy". >The part I find most promising is what this does t…

>>if a model is factually wrong a claim with a source is checkable and a claim from weights isn't.

>Why? If a model's weights claim that Bart Simpson became President in 2020, why does this fact suddenly become uncheckable?

Because in one case you have a source you can use to validate the fact, and in the other you don't. Though, as you explain earlier in your comment, the premise is misguided/hallucinated.

Re: Models Are Getting Dumber on Purpose

#135

Post by SoftTalker »

kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

If all the model knows is basic coding and Swift, then that's how you'll have to talk to it. At that point you might as well write the code yourself.

Re: Models Are Getting Dumber on Purpose

#136

Post by mohamedkoubaa »

pton_xd wrote: post #42
kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

> My model doesn't need to know a single line of python. If I had to guess, the weights necessary to encode "how to program" are much larger than the final step of "output python."

Sure but I bet this doesn't translate as well in this way to things like Erlang

Re: Models Are Getting Dumber on Purpose

#137

Post by SCHiM »

jiggawatts wrote: post #133
kennywinker wrote: post #128

Earlier quoted context omitted.

Those assumptions are just that - assumptions. "Local council in XYZ location" implies a bunch of things, and each one might be wrong for my specific circumstances. What better way to guide expectations than importing specific knowledge? I.e. if I import the english and catalan modules, then I probably want to localize my site in english and catalan. It would be trivial to have a pre-flight convo with an llm to guide…

See: https://en.wikipedia.org/wiki/Bitter_lesson Everyone assumes that carefully crafting a specific AI architecture with bits and pieces bolted together based on their human intuition is necessarily superior to simply using a bigger monolithic AI model. It turns out that the opposite is true, and has been demonstrated over and over again. The bitter lesson is this: You can simply ask a frontier model to do the thing…

>I get it. You don't feel ownership over someone else's AI. You don't feel involved, you don't feel like you have agency.

You don't _have_ ownership of someone else's ai, and that comes with real risks.

Security risks, privacy risks, business risk.

They might rug pull you, they might charge you more, or like atrophic, silently corrupt the answers, or code...

The labs are happy to jump on any emergent capability the scaling and training impart: generate prose, teach you things, cyber security, design, code, etc.

Do you really think that the frontier labs won't turn a popular capability, or trend they notice, into a first party tool if the ROI seems there? If it's your own private ai in your datacenter, you can keep it all secret, and not lose your business.

On the bitter lesson you're right of course:), wish I had a super computer to just scale that instead.

Re: Models Are Getting Dumber on Purpose

#138

Post by jan_m_savage »

znnajdla wrote: post #70
jimmaswell wrote: post #58

Earlier quoted context omitted.

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese p…

Thanks for putting this so well. The mathematical evidence for this “general intelligence underlying everything” is the “universal geometry of embeddings” paper. Fascinating read. Or as the ancient philosophers used to say, the one who knows God knows everything.

[dead]

Re: Models Are Getting Dumber on Purpose

#139

Post by jatora »

weitendorf wrote: post #119
jimmaswell wrote: post #58

Earlier quoted context omitted.

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese p…

This is an oversimplification: more data makes models smarter ceteris paribus, but mostly only because auto-regressive training (where most of the general knowledge comes from) is essentially compressing information that can be recalled later if it’s useful (or not recalled). Obviously there are differences in kind within “more data” too, you would much rather have all the books and blog posts in the world than all t…

[flagged]

Re: Models Are Getting Dumber on Purpose

#140

Post by markasoftware »

dist-epoch wrote: post #94
margalabargala wrote: post #91

Earlier quoted context omitted.

I agree with everything you say except this: > You can make any modern LLM explain its reasoning You can make any modern LLM create a plausible, self-consistent explanation that looks like reasoning, but it's not "the reasoning it used to arrive at that answer".

This was beautifully shown by asking a model to explain how it added two numbers together (something like 45+21), and it told a plausible story, when in fact they showed it was some rotation on a helix living in some internal manifold. Like asking a human "how did you catch that fast ball coming at you?"

It could be that the rotation in the helix manifold whatever is a low level representation of the logical steps (carry the 2, add the next column,...) it's describing. The point stands that the explanation it generates doesn't necessarily in all cases reflect what it "actually did" but your counterexample doesn't hold.
Post reply on HN