Earlier quoted context omitted.
I do think it's easy, in this technology discussion bubble in which we dwell, to overestimate the centrality of LLMs to the arc of developments in our time. They'll be important, but I don't think they'll be _that_ important, because the rest of society and the economy don't move at the speed of SV. Instead, they'll be overtaken by other, more traditional categories of events, ruptures and dislocations. Moreover, fol…
Functionally speaking, current models are intelligent by any reasonable definition of the term. You can test them on their understanding of complex domains such as software systems, and within that domain as an example, you can judge them on their ability to diagnose bugs, fix bugs, explain code to humans, design systems given vague specifications, and implement new systems. On all these tasks, current models are mea…
Furthermore, LLMs clearly do not "reason", despite the marketing around this term; their "chains of thought" are the nothing more than the result of having been trained on explicitly verbalised multi-step processes. There are many cases where the putative result arrived at in the chain of thought does not match the result emitted.
Whether they are "better" at software development than I am greatly depends on whether one is asking them to retrace worn technology paths that are well-represented in their training--in effect, to copy prior art--or to do something in quite obscure technology, or something quite novel altogether. (However, I will happily concede that most everyday business programming involves neither.)
Still, if LLMs were actually intelligent, let alone superhuman in the sense you suggest, then we would expect major scientific breakthroughs to be raining from the sky. If, say, an Einstein, could transform physics with only the knowledge gleaned from a human's feeble capacity to retain the literature of the time, I'd expect LLMs, who retain orders of magnitude more information with far greater fidelity and precision, to have offered at least a small slither of evidence of their superhuman capabilities.
I would also expect the objective progress and capabilities of this galaxy brain to be accelerating, not substantially slowing down as it has. GPT-2 to GPT-3 was truly a quantum leap, GPT-3 to GPT-4 was a substantial jump, GPT-4 to GPT-5 was meh, 5+ is basically unimportant, and so it goes for the other models. There are, of course, holes plugged and benchmarks where these evolutions have been, in various niche ways, consequential, but in the plainspoken meaning of model capability, the low-hanging fruit of pretraining was clearly exhausted quite some time ago. The carnival has been running on "agentic" / MCP / RAG / tool-use fumes since. This is moderately impressive and adds quite a bit of runway, but intelligence it is not.