Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Models Are Getting Dumber on Purpose

w4g1.dev

Post reply on HN
Print view View original

141–150 of 197 posts

Re: Models Are Getting Dumber on Purpose

#141

Post by weitendorf »

znnajdla wrote: post #59
kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

This is a fundamental misunderstanding of how LLMs work. You can’t really specialize a model. You specialize the harness. A well-trained general purpose LLM doesn’t need examples in its training data, it can write good code in a new language you invented yesterday with just a spec definition. And it will perform better than a small model trained on lots of examples of your invented language. The reason is because of…

You 100% can finetune or adapt/build on top of models, and specialize them or extend their capabilities. That’s literally what post training is.

The problem is that “finetuning” was a 2023 AI FOTM associated with products/demos that were almost exclusively using it for LLM character role-play/output style purposes (ie not in actual systems where they served a more functional role).

This made people think you could train models without replay/real evals by yoloing it with SFT (this is partially an artifact of that era being much heavier on autoregressive training and not so much evals). You really can finetune and get results but you have to treat it like a small ML training run, with real evals, and more intentionality than just “more examples”.

You can find pretrained and -instruct models on huggingface that clearly demonstrate what specialization/staged training runs do.

I’d be very wary of conflating finetuning with specialization/extending a model’s capabilities in general.

Re: Models Are Getting Dumber on Purpose

#142

Post by bigfishrunning »

znnajdla wrote: post #70
jimmaswell wrote: post #58

Earlier quoted context omitted.

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant. The model finds a structure to the problem you give it in a largely language-agnostic way that benefits from training on every language (these things are direct descendants of Google Translate), and even non-programming knowledge - the structure of your task might resemble an ancient Chinese p…

Thanks for putting this so well. The mathematical evidence for this “general intelligence underlying everything” is the “universal geometry of embeddings” paper. Fascinating read. Or as the ancient philosophers used to say, the one who knows God knows everything.

"the one who knows god knows everything" could also be a reference to the fact that god is unknowable, and therefore so is everything...

Re: Models Are Getting Dumber on Purpose

#143

Post by inigyou »

KerrAvon wrote: post #47

> the 24GB card that's been sitting in gaming PCs since 2022. I'd wager most people have less. In 2022 a 3080 might have 12 GB if you were lucky, 10 if you weren't -- and you paid for the privilege. A current RTX 5080 is only 16GB.

In 2021 I was considering getting a then-old K80 card because it had the highest VRAM on the market at 24GB and all newer cards had 8 or 12GB.

(Apparently this is because the K80 is two separate GPUs on one card, but I still think it counts if you only have one slot to put it in)

Re: Models Are Getting Dumber on Purpose

#144

Post by claiir »

COAGULOPATH wrote: post #85

This AI generated post (100% on Pangram) is pretty out of date. >On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions. SimpleQA hasn't been updated in a long time. Gemini 2.5 Pro is a sixteen-month-old model, not "the best recall money can buy". >The part I find most promising is what this does t…

Yea the "When the fact lives outside the model, a wrong answer has an address" sentence seems aggressively AI written. Saw that and my senses went off.

Re: Models Are Getting Dumber on Purpose

#145

Post by Muromec »

stanac wrote: post #43
kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

Sounds like MoE but more pluggable. Not sure if anyone is researching something like this. I still think your swift or GIS model will need basic reasoning and coding to work, so it's more like multiple smaller models which you can load as needed, e.g. sub-agents for GIS needs the GIS model. Another approach would be to have basic coding and reasoning model and then load specification for language and libraries into c…

I would, once it is possible, specifically unplug the shitcoder expert that sometimes shows up and puts `as unknown as` and bypasses the commit hook that is supposed to enforce it. That should be somewhere near java coder expert in the whatever dimensional space. I can feel it being true.

Re: Models Are Getting Dumber on Purpose

#146

Post by 8bitbeep »

jiggawatts wrote: post #133
kennywinker wrote: post #128

Earlier quoted context omitted.

Those assumptions are just that - assumptions. "Local council in XYZ location" implies a bunch of things, and each one might be wrong for my specific circumstances. What better way to guide expectations than importing specific knowledge? I.e. if I import the english and catalan modules, then I probably want to localize my site in english and catalan. It would be trivial to have a pre-flight convo with an llm to guide…

See: https://en.wikipedia.org/wiki/Bitter_lesson Everyone assumes that carefully crafting a specific AI architecture with bits and pieces bolted together based on their human intuition is necessarily superior to simply using a bigger monolithic AI model. It turns out that the opposite is true, and has been demonstrated over and over again. The bitter lesson is this: You can simply ask a frontier model to do the thing…

> It's the classic "But I can customise EMACS endlessly, why would I use an actual IDE?" argument all over.

That paragraph sets me off. I’ll take Vim and Emacs over VSCode and Eclipse any day.

Re: Models Are Getting Dumber on Purpose

#147

Post by nojs »

COAGULOPATH wrote: post #85

This AI generated post (100% on Pangram) is pretty out of date. >On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions. SimpleQA hasn't been updated in a long time. Gemini 2.5 Pro is a sixteen-month-old model, not "the best recall money can buy". >The part I find most promising is what this does t…

> This AI generated post (100% on Pangram) is pretty out of date.

Quite ironic given the topic. It seems that the author’s model indeed contained too much knowledge about old Gemini releases, and did not do enough tool calling.

Re: Models Are Getting Dumber on Purpose

#148

Post by leecoursey »

Are we slowly discovering what evolution already "realized" via optimization of human brains? i.e., procedural knowledge is easier to maintain than a library of facts, and facts can be accessed in external repositories if one knows the procedure for doing so.

Re: Models Are Getting Dumber on Purpose

#149

Post by kentonv »

malfist wrote: post #93
COAGULOPATH wrote: post #85

This AI generated post (100% on Pangram) is pretty out of date. >On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions. SimpleQA hasn't been updated in a long time. Gemini 2.5 Pro is a sixteen-month-old model, not "the best recall money can buy". >The part I find most promising is what this does t…

Seriously, anybody with a passing knowledge of LLMs knows thats not how they function. You can't encode logic in them because that's not how they work. It's a statistical model with useful emergent properties. It doesn't think, it doesn't reason, it isn't aware of facts or the rules of logic.

> It's a statistical model with useful emergent properties. It doesn't think, it doesn't reason, it isn't aware of facts or the rules of logic.

What makes you so sure your own brain doesn't work the same way?

Re: Models Are Getting Dumber on Purpose

#150

Post by zmmmmm »

It's a nice idea conceptually to decouple knowledge from reasoning but like everything in life I think it's something of a fantasy. When you ask a model to do something, its response is grounded in all the world knowledge it has from those facts. If I ask it to color a circle blue and then red, it knows what blue and red are, that they are colors, that they are different, and this is essential to complete the reasoning about that task. The idea of abliterating all the "knowledge" but not affecting the reasoning pretends there is a clean boundary between them that isn't there. Yes if you take it to the extreme - does it "know" Einstein's birthday - it is very stark. But I think the boundary to where general knowledge starts to interplay with broad reasoning is not nearly as far from those specific facts as people assume.
Post reply on HN