Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Models Are Getting Dumber on Purpose

w4g1.dev

Post reply on HN
Print view View original

31–40 of 197 posts

Re: Models Are Getting Dumber on Purpose

#31

Post by cyanydeez »

kennywinker wrote: post #13
msdz wrote: post #9

Great article, even if it will be interesting to see whether things continue to develop in such a direction or not. > There's a version of this future where the model card stops listing a knowledge cutoff at all, because what's left in the weights goes stale on a scale of years instead of weeks. Future? Even just recently I’ve read of two approaches to this problem: Cactus have come up with Needle [0][1], which is th…

That kind of setup is super dependent on a search engine, and search keeps getting worse.

the technicals of search did not get worse. the economics of the search provider did. If government wasn't stuck in the 50s mindset, search provider would have been nationalized just like a utility.

Re: Models Are Getting Dumber on Purpose

#32

Post by conartist6 »

kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

This is going to be how I write my resume now: how many billion parameters of knowledge I have on each topic

Re: Models Are Getting Dumber on Purpose

#33

Post by pulkitsh1234 »

But is Reasoning and Facts truly separate ?

To reason properly about the human condition (eg. World War) wouldn't you need to reason on some facts ? And then reason how some "facts" change the human behaviour ? How can you arrive via pure reasoning to predict how a collective of humans act ? We are not reasonable, humans are not logical deterministic machines confined to algebraic rules.

Re: Models Are Getting Dumber on Purpose

#34

Post by kennywinker »

kalb_almas wrote: post #22
kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

I would love this but I think the General Reasoning and Make No Mistakes modules would be massive.

I mean, I get pretty damn good general reasoning out of Qwen3.8-27B quantized to 4bits, and that knows Swift, Python, Node, Ruby, Rust, etc. etc. etc.

And so far even the biggest model doesn't seem to have a working Make No Mistakes module, so maybe that's not needed

Re: Models Are Getting Dumber on Purpose

#36

Post by jeremyloy_wt »

kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

Admittedly I’m pretty ignorant of the details, but I thought this was the mixture of experts architecture

Re: Models Are Getting Dumber on Purpose

#37

Post by nvme0n1p1 »

scld wrote: post #24
edgyquant wrote: post #18

Earlier quoted context omitted.

This would defeat the AGI narrative/belief that so many building these models have

On the otherhand, your own brain probably doesn't use your eyes to hear.

You don't think reading lips helps you understand people?

Re: Models Are Getting Dumber on Purpose

#38

Post by alightsoul »

kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

This is possible today with an agent such as OpenClaw and hermes

Re: Models Are Getting Dumber on Purpose

#39

Post by hkt »

kennywinker wrote: post #4

Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…

I strongly suspect this will be the future

Re: Models Are Getting Dumber on Purpose

#40

Post by lilerjee »

> This mostly solves hallucination

Current AI is like the film company producing TV series or movies

Your question is like a story outline. You tell the film company that this is the movie you want. The AI film company then searches for existing similar stories. If similar stories do not exist or details are missing, screenwriters use imagination to fill in the gaps (remember hallucination? It's just a makeup.)

So you cannot solve hallucination of AI

Post reply on HN