Earlier quoted context omitted.
> These things are "only" next token/word guessers. This is precisely the kind of vacuous "this is technology, I know technology, this is simple" hubristic underestimation that's being called out. There is no upper bound to the intelligence of a "next token/word guesser". You can end up incorporating an entire world model to your predictions to improve their accuracy, and arguably this has already happened, to a curr…
"This is precisely the kind of vacuous ..." So I am devoid of anything? Nice. I slapped "only" within quotes to imply that there is more going on and a lot more complexity than implied by a naked reading of my comment. I'm sorry you missed that. There is no notion of a bound or even intelligence for a LLM. It is a tool and no more - we know how they work - that is defined and we run our own. We can marvel at what loo…
The completion was "vacuous [..] underestimation". Something you appeared to be doing in this one phrase you wrote, not something you are. I don't know anything about you as an entire person. Please try not to take criticism of some of your written thoughts so personally. I continue to take issue with characterizing the LLM as "a word guesser" because it implies a limit to capability that I don't think actually exists.
> There is no notion of a bound or even intelligence for a LLM.
When LLMs are outscoring humans on many/most standardized tests, including for tests where the questions are novel, I also disagree that there is "no notion of intelligence for an LLM". It feels like goalpost-moving, to the extent that I now have no idea what you actually mean when you say intelligence.
> we know how they work
I think this is also a hubristic statement. The researchers working on these systems do not speak like this. They say things like "we did reinforcement learning on question-answering in English and it turns out it answers questions in French too now and we were surprised and can't explain why that happens".