Earlier quoted context omitted.
Will it improve though? I’m not a GPT hater or denier, but how do you even predict that it hasn’t already hit the wall? They can increase parameter count x100 again, but correctness is not some knob they can just dial up to 10. What if a learning dataset simply has not enough info for a correct answer to have a greater weight than all the “con” noise? What if an answer requires a sort of reasoning inaccessible to LLM…
> What if a learning dataset simply has not enough info for a correct answer to have a greater weight than all the “con” noise? Indeed. I wonder what happens as available training data shifts from purely human-generated (now) to largely AI-generated (soon). Is this an information analogue to the “gray goo” doomsday that an uncontrolled self-replicating nano device could cause? >can it answer “I don’t know this” Such…
A lot of models were trained to provide some quantifiable output 100% of the time, even if that output was wrong. Ie image recognition models "82.45% certain that is a dog", whereas it makes _all_ the difference for it to be able to say "82.42% certain that is a dog and 95.69% certain I don't know what that is" to indicate that the image has many features of a dog, but not enough for it to be more certain that it is a dog than isn't. It's the negative test problem I guess; us devs often forget to do it too.
In a way I wonder if that's how some of the systems in our brains work as well; ie we evolved certain structures to perform certain tasks, but when those structures fail to determine an action, the "I don't know" from that system can kick back into another. Thing like the fear response: brain tries to identify dark shadow & can't, kicks back to evolutionary defence mechanisms of be scared/cautious feel fear as it's saved the skins of our forebears.