Earlier quoted context omitted.
You won't have to, it's already happening.
Let's wait till AI makes hiring decisions.
"None of the human candidates were qualified. You should hire more computers."
IQ tests results for AI
101–110 of 359 posts
Re: IQ tests results for AI
#102Wow. This matches my experience pretty closely. Haven't put GPT-5 Pro through its paces much yet, but these numbers suggest I should.
Re: IQ tests results for AI
#103Well, the site owner works for Stossel TV and Stossel is a "rationalist" AI apologist:
https://www.johnstossel.com/ai-is-coming-for-your-job/
All standard talking points on one page (he forgot lump of labor though). So I'd take this test with a big grain of salt.
Re: IQ tests results for AI
#104Earlier quoted context omitted.
They’re definitely going to overfit on this, but this will be much better from a marketing perspective. Normies don’t know wtf an MMLU is, but they do know what IQ is and that 140 is a big number. Can’t wait for CEOs to start saying “why would we hire a 120 IQ person who works 9-5 with a lunch break when we can hire a 170 IQ worker who works 24x7 for half the cost??”
"Workers rejoice as model overfitted to score 170 on IQ test turns out to be incapable of performing basic tasks..."
Nothing matters except the first couple of time derivatives. The workers aren't getting any better.
Re: IQ tests results for AI
#105AI has a 140 “IQ” but understands nothing. That’s because AI does not understand anything: it just predicts the next token based on previous tokens and statistics. AI can give me five synonyms for any Latin word, because that’s just statistics, and it can regurgitate rules about metrical length of syllables, but it can’t give me synonyms matching a particular metrical pattern, because that would involve applying know…
That’s because AI does not understand anything: it just predicts the next token based on previous tokens and statistics
As opposed to what you were doing when you wrote that.
Re: IQ tests results for AI
#106How many of the answers were in the training data?
Isn't this like saying that a spellchecker is "very smart" because it did well at a spelling bee? It isn't, it just has a list of answers.
Re: IQ tests results for AI
#107Earlier quoted context omitted.
That seems intuitive to me, but lots of other things in science seemed intuitive because I wanted to believe them. If the measured IQ difference in individuals can be overwhelmed by simple factors like “have I taken the test before”, we don’t really have a useful empirical measurement to say these things and we’re just stating our hopes and dreams.
The training effect in test-retest is dependent on g as well. It is intelligent to learn from past experiences. Measuring g is hard and taking shortcuts is tempting. A reasonable repeatable g factor test takes hours, and is too often replaced by a single test. There are ways around the test-retest issues but they are roads less travelled.
My high school was right across from a branch of a university (UHD) where the PhD candidates developed IQ tests. We (the HS students) could take them for extra credit. My favorite example was a block-arranging test (there was a set of blocks & some pictures). Anyways, they printed the blocks "symmetrically"; once I figured that out, making the picture was limited only by how quickly I could move. (The test normally had you looking at all sides of the cube, repeatedly.) My "IQ" was well over 200 on that test. The candidate said that it was going to set their lab back bag years.
Re: IQ tests results for AI
#108Earlier quoted context omitted.
Oh, the irony of that comment...
Hardly any...
The irony isn't that the GP is right, because they are. "AI does not understand anything: it just predicts the next token based on previous tokens and statistics."
The irony is that we've recently learned that "just predicting the next token" is good enough to hack code, compose music and poetry, write stories, win math competitions -- and yes, "give synonyms matching a particular metrical pattern" (good luck composing music and poetry without doing that) -- and the GP doesn't appreciate what an earthshaking discovery that is.
They are too busy thumping their chest to assert dominance over a computer, just as any lesser primate could be expected to do.
Re: IQ tests results for AI
#109How much of this is because so many IQ tests are part of the training data?
Re: IQ tests results for AI
#110Big caveat here: This website's method doesn't work at all for humans the way it works for LLMs. For humans, there is a strict time limit on these IQ tests (at least in officially recognised settings like Mensa). This kind of sequence completion is mostly a question of how fast your brain can iterate on problems. Being able to solve more questions within the time limit means you get a higher score because your brain…
The point of this is not so much to compare humans with AI. But to compare AI with other traditional software development approaches to solve this domain (IQ test, in this case). I believe, and I could be wrong, it will be nearly impossible, or too expensive, to develop deterministic software to beat AI in IQ test.