I think the biggest problem with the models is they don’t actually have any decent lookups except chunked document embedding search
Models Are Getting Dumber on Purpose
41–50 of 197 posts
Re: Models Are Getting Dumber on Purpose
#42Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…
> My model doesn't need to know a single line of python.
If I had to guess, the weights necessary to encode "how to program" are much larger than the final step of "output python."
Re: Models Are Getting Dumber on Purpose
#43Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…
Sounds like MoE but more pluggable. Not sure if anyone is researching something like this. I still think your swift or GIS model will need basic reasoning and coding to work, so it's more like multiple smaller models which you can load as needed, e.g. sub-agents for GIS needs the GIS model.
Another approach would be to have basic coding and reasoning model and then load specification for language and libraries into context, it could work for self-hosted models, but I don't want whole specification of the language to be send to API and waste tokens on that.
Re: Models Are Getting Dumber on Purpose
#44Intelligence vs Knowledge LLMs work is being intelligent not having knowledge of everything is ok. But, they have to be intelligent enough (with some degree of knowledge) that where to find the information (search tools or any other tools for that matter)
Not too long ago, the claim was that having multiple Phds worth of knowledge was the real value of AI, but now that's gone and it's suddenly a good thing.
That's a sales tactic -- not a logical position.
Re: Models Are Getting Dumber on Purpose
#45I think the idea is reasonable, however the SimpleQA Bench stopped measuring in Sept. 2025. So newer data would be interesting. (It seems a bit like an AI generated argument that uses old facts - something that happens to me quite often)
There’s SimpleQA Verified
Re: Models Are Getting Dumber on Purpose
#46Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…
Admittedly I’m pretty ignorant of the details, but I thought this was the mixture of experts architecture
No, that’s a common misconception. MoE despite the name doesn’t have an “expert” slice that’s an expert in any particular thing. It’s just trained models such that a subset of all memory weights is active on a single token. Often, the weights active on the next token are different. It’s used to improve memory bandwidth (throughput) and also to even out load in a distributed serving model - one GPU hosts one “expert” and the conversation is routed to it for the next token. This requires a lot of movement of the KV cache of course so it’s difficult to retain speed across multi node splits - usually it’s a rack of GPUs that you do this for.
Re: Models Are Getting Dumber on Purpose
#47> the 24GB card that's been sitting in gaming PCs since 2022.
I'd wager most people have less. In 2022 a 3080 might have 12 GB if you were lucky, 10 if you weren't -- and you paid for the privilege. A current RTX 5080 is only 16GB.
Re: Models Are Getting Dumber on Purpose
#48Ideally what I'd like to see is pluggable knowledge bases. So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python. Then when I want to research electronics components, I grab a 15B model of agentic research techniques…
Sounds like MoE but more pluggable. Not sure if anyone is researching something like this. I still think your swift or GIS model will need basic reasoning and coding to work, so it's more like multiple smaller models which you can load as needed, e.g. sub-agents for GIS needs the GIS model. Another approach would be to have basic coding and reasoning model and then load specification for language and libraries into c…
My motivation is somehow the model should be able to work without hitting an external resource every single time and somehow have all the knowledge necessary locally so we don't get rate limited but how can you stay up to date with the latest information while still remaining a good boy scout by not reaching out to tool use and scraping every single GitHub repo and issue every single time?
Re: Models Are Getting Dumber on Purpose
#49[deleted]
Re: Models Are Getting Dumber on Purpose
#50> This mostly solves hallucination Current AI is like the film company producing TV series or movies Your question is like a story outline. You tell the film company that this is the movie you want. The AI film company then searches for existing similar stories. If similar stories do not exist or details are missing, screenwriters use imagination to fill in the gaps (remember hallucination? It's just a makeup.) So yo…
Indeed !
LLM are creative writers, not journalists.
Relying on overfitting for factual accuracy in not tenable.
I don't understand why grounded RAG with judges in not the norm.