IQ tests results for AI • Hacker News Board

Re: IQ tests results for AI

#21

Post by krapp » Sun, Aug 17, 2025, 10:12 AM UTC

gpt5 wrote: post #6

The way human IQ testing developed is that researchers noticed people who excel in one cognitive task tend to do well in others - the “positive manifold.” They then hypothesized a general factor, “g,” to explain this pattern. Early tests (e.g., Binet–Simon; later Stanford–Binet and Wechsler) sampled a wide range of tasks, and researchers used correlations and factor analysis to extract the common component, then norm…

I imagine the value of something like this is for business owners to choose which LLMs they can replace their employees with, so it using human IQ tests is relevant.

Re: IQ tests results for AI

#22

Post by sigmoid10 » Sun, Aug 17, 2025, 10:12 AM UTC

Big caveat here:

This website's method doesn't work at all for humans the way it works for LLMs. For humans, there is a strict time limit on these IQ tests (at least in officially recognised settings like Mensa). This kind of sequence completion is mostly a question of how fast your brain can iterate on problems. Being able to solve more questions within the time limit means you get a higher score because your brain essentially switches faster. But for LLMs, they just give them all the time in the world in parallel and see how many questions they can solve at all. If you look at the examples, you'll see some high end models struggling with some the first questions, that most humans would normally get easily. Only the later ones get hard where you really have to think through multiple options. So a 100 IQ LLM in here is not technically more intelligent in IQ test questions than 50% of humans.

If anything, this shows that some LLMs might win against humans because they can spend more time thinking per wall clock time interval thanks to the underlying hardware. Not because they are fundamentally smarter.

Re: IQ tests results for AI

#23

Post by tgv » Sun, Aug 17, 2025, 10:13 AM UTC

pzmarzly wrote: post #17

Snapshot: https://archive.ph/0ihF5

What's their obsession with clock when there is only one hand? I guess there isn't training material with similar shapes describing them as angles. Even a compass would make more sense.

Re: IQ tests results for AI

#24

Post by yetihehe » Sun, Aug 17, 2025, 10:14 AM UTC

FranOntanaya wrote: post #18

diggan wrote: post #11

Earlier quoted context omitted.

I mean not really, as always you just need to make sure you're not doing 10s of dynamic calls for each page load and if you do, add some minute-long cache at least. Most of the stuff that gets hugged to death really shouldn't, most of the times it's just static content that is trivial to host on even $10/month instances.

The amount of calls on some pages displaying the simplest stuff is mind-boggling. 160 requests for a page just displaying a HTML5 video and a title, 360 requests for a Reddit page, it's nuts. We don't need to be like this.

"We and our 350 partners care about your privacy".

Re: IQ tests results for AI

#25

Post by Telemakhos » Sun, Aug 17, 2025, 10:15 AM UTC

AI has a 140 “IQ” but understands nothing. That’s because AI does not understand anything: it just predicts the next token based on previous tokens and statistics. AI can give me five synonyms for any Latin word, because that’s just statistics, and it can regurgitate rules about metrical length of syllables, but it can’t give me synonyms matching a particular metrical pattern, because that would involve applying knowledge. If I challenge its wrong answer, it will apologize and give me further wrong answers that are wrong in the same way, because it cannot learn.

Re: IQ tests results for AI

#26

Post by bitwize » Sun, Aug 17, 2025, 10:15 AM UTC

mirekrusin wrote: post #19

codr7 wrote: post #9

Earlier quoted context omitted.

You won't have to, it's already happening.

Let's wait till AI makes hiring decisions.

You won't have to, it's already happening.

Re: IQ tests results for AI

#27

Post by olalonde » Sun, Aug 17, 2025, 10:18 AM UTC

Telemakhos wrote: post #25

AI has a 140 “IQ” but understands nothing. That’s because AI does not understand anything: it just predicts the next token based on previous tokens and statistics. AI can give me five synonyms for any Latin word, because that’s just statistics, and it can regurgitate rules about metrical length of syllables, but it can’t give me synonyms matching a particular metrical pattern, because that would involve applying know…

Oh, the irony of that comment...

Re: IQ tests results for AI

#28

Post by YetAnotherNick » Sun, Aug 17, 2025, 10:20 AM UTC

gpt5 wrote: post #6

The way human IQ testing developed is that researchers noticed people who excel in one cognitive task tend to do well in others - the “positive manifold.” They then hypothesized a general factor, “g,” to explain this pattern. Early tests (e.g., Binet–Simon; later Stanford–Binet and Wechsler) sampled a wide range of tasks, and researchers used correlations and factor analysis to extract the common component, then norm…

ARC-AGI challenge aims for that. In fact the objective is even more strict that the tasks must be trivial for most humans given time.

Re: IQ tests results for AI

#29

Post by roenxi » Sun, Aug 17, 2025, 10:21 AM UTC

https://www.trackingai.org/political-test is almost the more interesting part of the website, there is a surprising uniformity of left-libertarian political views.

Even assuming that companies prune out authoritarianism from their models for whatever reason, surely we'd expect at least one of them to drift over into mild economic right-wing territory. It'd be interesting to know what is causing that bias.

Re: IQ tests results for AI

#30

Post by eqvinox » Sun, Aug 17, 2025, 10:21 AM UTC

stared wrote: post #13

My take is that it’s easier to train a model to ace short, low-context tasks like IQ tests. That doesn’t necessarily transfer to more complex reasoning. While on the Mensa Norway test GPT-5 gets over 140, on an offline test it goes down to ~120. It is interesting to look at the political spectrum as well ( https://www.trackingai.org/political-test ) - ar are liberals, even Grok 4. The political leaning isn’t surprisi…

> While on the Mensa Norway test GPT-5 gets over 一四, on an offline test it goes down to ~一二.

Since IQ tests are fundamentally timed, those numbers are meaningless to compare with human numbers. Or maybe dangerous since it's hard to de-context them even if you know that. Hence my cheeky 漢字.

(Yes they might be useful to compare LLMs with each other, but that is outstripped by the risk of misreading it against what we know as "IQ".)