Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Models Are Getting Dumber on Purpose

w4g1.dev

Post reply on HN
Print view View original

191–197 of 197 posts

Re: Models Are Getting Dumber on Purpose

#191

Post by springtimesun »

I like this approach and I would be pretty happy if this is the way models go, but my question is: go look it up where? Search quality is declining for multiple reasons and I don’t see it becoming better. At best it might be able to outrun the slop, but that means standing still relative to quality now. If we are talking about technical facts or something that can be looked up in docs, sure, but factual questions at search time just defer the (hopefully) considered choice of what to include in weights with the ad hoc decision of what to include from the search results. At least if it’s in the weights then model has a prior to weight the evaluation of sorting through the results.

Maybe that’s an overly negative take. Maybe the search engines will be able to find quality. Maybe humans will continue to write quality. Maybe the models can reason their way in to quality from first principles.

Re: Models Are Getting Dumber on Purpose

#192

Post by kennywinker »

Ohentis wrote: post #178
8bitbeep wrote: post #146

Earlier quoted context omitted.

> It's the classic "But I can customise EMACS endlessly, why would I use an actual IDE?" argument all over. That paragraph sets me off. I’ll take Vim and Emacs over VSCode and Eclipse any day.

Well vscode is also not an actual ide

https://en.wikipedia.org/wiki/Visual_Studio_Code

> Visual Studio Code (commonly referred to as VS Code)[11] is an integrated development environment

Unless you were just going for a sick burn on vscode, in which case carry on :)

Re: Models Are Getting Dumber on Purpose

#193

Post by kennywinker »

c7b wrote: post #182
kennywinker wrote: post #129

Earlier quoted context omitted.

With what I know about how LLMs work now, I guess I am suggesting more specific variants. Qwen3.8 has a 2.4T version and a 27B version. I understand that to mean that they are the same architecture, just one version has a massive training set and the other has a very small subset. So, it seems very possible that variants of 27B could be generated that tune it for specific things by selecting different training data f…

> I understand that to mean that they are the same architecture, just one version has a massive training set and the other has a very small subset. No. It means that the one model has 2.4 trillion parameters while the other has only 27 billion. I don't know the details about their architecture or training, but presumably they used the same or similar training sets for both and a conceptually similar architecture, sca…

> it'll be outgunned by something that just leverages raw computation better.

The issue for me is that the raw computation is coming at the cost of the planet. Throwing an aircraft carrier at a problem that needs a bicycle is dumb, but because the damage to the environment required to scale up computation isn't included in the price of that computation - it's easier to just toss the aircraft carrier at every little problem.

So when I say I want to pick and choose, and use smaller models, it's because I like technology and I don't want to hate LLMs, but I also like the planet and don't want LLMs to continue to accelerate environmental collapse.

Re: Models Are Getting Dumber on Purpose

#194

Post by c7b »

kennywinker wrote: post #193
c7b wrote: post #182

Earlier quoted context omitted.

> I understand that to mean that they are the same architecture, just one version has a massive training set and the other has a very small subset. No. It means that the one model has 2.4 trillion parameters while the other has only 27 billion. I don't know the details about their architecture or training, but presumably they used the same or similar training sets for both and a conceptually similar architecture, sca…

> it'll be outgunned by something that just leverages raw computation better. The issue for me is that the raw computation is coming at the cost of the planet. Throwing an aircraft carrier at a problem that needs a bicycle is dumb, but because the damage to the environment required to scale up computation isn't included in the price of that computation - it's easier to just toss the aircraft carrier at every little p…

It's a noble cause, but there are probably bigger levers to pull than the model size if you care about environmental impact.

If you're running Qwen3.8-27B on energy-efficient hardware like a Mac or a DGX Spark instead of an API (likely running on H100s), I'm sure you're having much more of an impact than you would by switching to, say, a 9B coding-only model on the same hardware. The thing is, I think you won't be able to go orders of magnitude smaller, because a lot of the usefulness of LLMs comes from emergent smartness, and you typically need a minimum amount of complexity to see such emergent phenomena (and I think we're pretty far from understanding this kind of emergence, much further than from the next model generation that annihilates the current one on benchmarks yet again).

Re: Models Are Getting Dumber on Purpose

#195

Post by janpeuker »

I think Opus 5 is the best example of this - "relentlessly proactive" agent, therefore reducing hallucinations, but at the expense of fast, factual answers and direct logical paths. Everything becomes a pamphlet compiled from first principles.

Re: Models Are Getting Dumber on Purpose

#196

Post by RealWed6 »

RealWed6 wrote: post #155
claiir wrote: post #144

Earlier quoted context omitted.

Yea the "When the fact lives outside the model, a wrong answer has an address" sentence seems aggressively AI written. Saw that and my senses went off.

Senses of what? LOL. The whole Internet is AI generated by now and we all contribute to that on daily basis. get used to it or dull your senses ...

Ah Ah Ah – how we did't like this. Downvoting me will surely help! Can't you see how much slope is already around? Don't we – you and me – contribute to that, especially at work? Isn't "dull your senses" standard answers of most expensive shrinks? So what did you disagree with? Or you simply didn't like the truth? Ok, I got it. No problem.
Post reply on HN