Live data from Hacker News

Models Are Getting Dumber on Purpose

w4g1.dev

51–60 of 197 posts

Re: Models Are Getting Dumber on Purpose

#51

Earlier quoted context omitted.

What is the rest of the world using LLMs for? Agentic stuff seems pretty universal. Obviously coding stuff is only interesting to people who want to code, but automating complex digital tasks seems useful in all kinds of contexts.

I’m working at a company where everyone is using LLMs for everything and I’m not aware of anybody using anything truly genetic. It’s all human in the loop babysitting the tool.

What tool?

Even basic clients are now harnesses. A lot of chat interfaces are using memory systems, web search and other stuff under the hood.

Not as agentic as openclaw, but not a straight closed conversation either.

Re: Models Are Getting Dumber on Purpose

#52

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

> So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. Aren't you describing RAG or even MCP servers? Heck, nowadays you get that also with agent skills and specialized tool calling.

I don’t think so because those both live in the context window and as such pollute it when they’re not performing optimally.

I think having unused or rarely used weights doesn’t influence the results as poorly as RAG injecting irrelevant facts.

It sounds to me like some sort of “dynamic MoE” where you can add/create or remove experts on the fly.

I think what you’re describing is the closest approximation we reasonably have right now though.

Re: Models Are Getting Dumber on Purpose

#53
post #42

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

> My model doesn't need to know a single line of python. If I had to guess, the weights necessary to encode "how to program" are much larger than the final step of "output python."

But the understanding of the language library ecosystem, or even better, your codebase, could let it execute faster and with less context usage.

Re: Models Are Getting Dumber on Purpose

#56

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

Admittedly I’m pretty ignorant of the details, but I thought this was the mixture of experts architecture

MoE's in the abstract often get presented as if theres explicit layers of experts for any given domain of knowledge, like your coding tasks are being routed to coding experts, but it's really not that at all.

THe original MoE paper from Noam Shazeer et al. is worth a read on this bit, though the paper is admittedly pretty dense. But TL;DR is that each expert layer is learning highly abstract, localized structural and syntactic patterns in the data to minimize the loss function, and its doing this token-by-token (which in some cases may have some domain clustering, but that's just incidental).

When you start batching your queries, even if they all seem like theyre in a single domain, if you visualized the activations you'd notice that most if not all of the network is lighting up on the batched forward pass.

Re: Models Are Getting Dumber on Purpose

#57
post #42

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

> My model doesn't need to know a single line of python. If I had to guess, the weights necessary to encode "how to program" are much larger than the final step of "output python."

ie what everyone asking for this fails to immediately realize.

Re: Models Are Getting Dumber on Purpose

#58

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese poem that influences the model's response, for example. That structure is considered a form of compression, as some fascinating and illuminating recent 3blue1brown videos get into - a common pattern in Haskell or FORTRAN and a situation described in an ancient Chinese poem may all compress to something quite similar to your task, thus when the model compresses the idea of your task it immediately draws from those ideas.

There are "experts" which do divide parts of the model that are found to activate together for specific tasks, so they can be processed in parallel to join the result at the end, but it's nowhere near the granularity of a SwiftUI expert and a python expert. The difference in those things is so trivial from an abstract point of view that it would make no sense. They would be 99% the same.

Distillations also come into this but I'm highly skeptical you could make one guaranteed to only know programming and only in one programming language (especially with as small a sample set as SwiftUI relative to something like C) without its efficacy being hobbled by tunnel vision. Reminiscent of the SpongeBob episode where he empties his mind of everything except fine dining and breathing, then can't remember his name and goes insane. Beyond the basic concepts of general coding and the trivia of syntax, getting anything done requires a large intersection of disparate world knowledge and the ability to apply it to new situations.

Re: Models Are Getting Dumber on Purpose

#59

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

This is a fundamental misunderstanding of how LLMs work. You can’t really specialize a model. You specialize the harness. A well-trained general purpose LLM doesn’t need examples in its training data, it can write good code in a new language you invented yesterday with just a spec definition. And it will perform better than a small model trained on lots of examples of your invented language. The reason is because of the “universal geometry of embeddings”, i.e all human languages have the same underlying pattern structure, so any model that is very good in any language is good in all languages. Attempting to specialize a model for a particular purpose often decreases overall performance. Fine-tuning is just a hack to make dumb models more reliable on limited tasks but they become incapable of doing anything else. Unless you are building a factory assembly line where a model is literally doing the same thing over and over, you almost always want a general purpose model over a specialized one.
Post reply on HN