The idea that intelligence isn't just statistics to me is the far more radical position here. If it's not statistical modelling then what is it? Intelligence is not magic. Any prediction requires some amount of probabilistic modelling.
That said, I think there is probably a significant amount of meta modelling required to achieve true AGI and it seems unlikely that current architectures can achieve this. The fact that LLMs don't seem to have inner thought or the fact that learning and inference is separate is a huge limitation of current algorithms.
It seems to me when you think about what us humans do the ability to meta analyse and pipe our thoughts through various processes in our head, then discard inaccuracies and adapt to new information is important. LLMs seem rigid in their thinking because they don't do this meta reasoning, and are completely unable to adapt to new information. Current LLM act kinda like humans do during exams. In an exam we feel we must provide an answer to every question and if we don't know the answer we'll just make something up. But outside of exams humans don't do this. If we don't know something we'll gather information, we'll test theories, we'll ask questions, then we'll adapt and take onboard new information.
LLMs don't do this and therefore feel rigid in their thinking and often act irrationally. For example, you can convince an LLM of something with some faulty information or logic then start a new chat and it's reverted back to whatever position it was giving before. They only really get better at reasoning when us humans walk them through meta reasoning processes, but it cannot do this itself and even when we help them with this they do not adapt in light of it.
What I will say is that we have clearly made one critical discovery towards AGI (and super intelligence) and that is that scale is important. We also seem to be making process on the algorithmic side too and are certainly edging closer to something that with enough scale could approach something that looks like AGI.
I'll also add what LLMs are able to do in their infancy is frankly incredible and it's not hard to imagine that it would only take a few additional algorithmic breakthroughs from here to get to something very close to AGI, if not achieve it. My guess is that we already have the scale and much of the base neural network architecture required. The main limitation in my opinion is the that training and inference are separate steps – not that LLMs use statistics.
Finally, no one serious that I'm aware of believes that simply scaling current iterations of LLMs will achieve AGI anyway. Dismissing the possibility of AGI because current LLM architecture is missing a few things seems both silly and an uncharitable position to me. The important disagreement here is just how many more algorithmic breakthroughs we need to achieve AGI. We all know GPT-4 reasoning ability is based too wholly in low-level statistical reasoning, and is incapable of the higher-level meta reasoning required for more advantaged intelligence. The real question is how far are we from making process on this.
Just my opinions anyway.