> Current LLMs are obviously far below the average adult human. Every demo I've ever seen brags about approaching the competency of an ordinary high school intern.
There's a lot of bragging, sure. But what I've seen from the OpenAI models is more like a university level intern or a fresh graduate.
Not that it matters, as the point is to aim for where the ball is going rather than where it is now, but I'd say that this makes them above the average human performance (at least, within its domain).
> Even if they reach above average our whole society is structured around managing billions of above average humans.
That's putting a binary cutoff in an arbitrary place; half the population is always above average on any measure (or indeed topic), what I wrote was "significantly above average".
To crudely approximate with IQ, which is a bad measure for humans and much worse measure for AI, there's just over a billion humans of IQ > 115, around 182 million of IQ > 130, and around 8 million of IQ > 145.
I'm not sure how much to trust estimates of politician's IQs given the tribalism involved, but surveys of other groups tend to show that around half of the highest performing leaders (CEOs etc) with the most power, are in the top 1% by IQ.
(Who knows when, or if, someone will make an AI that's good at the important tasks within of each of those roles).
> What's novel?
Two things:
1) Breadth: the current vogue is very general models, so they're interns sure (even though I rank them as a higher level of intern than you), but they are interns at everything — there is no human you can hire with a even a mediocre grasp of Mandarin and Arabic and Welsh and python and CSS and calculus and magnetohydrodynamics and psychology and …
2) Price: there is no human who can read a million tokens for a price equal to two days of the UN abject poverty threshold, or write a million tokens for six days of the UN abject poverty threshold.
You physically can't work that fast, even if you were typing [a] then [space] ten times a second for 18 hours a day; and you basically can't feed yourself if you earn less than that per day.