Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IQ tests results for AI

trackingai.org

Post reply on HN
Print view View original

271–280 of 359 posts

Re: IQ tests results for AI

#271

Post by perching_aix »

testdelacc1 wrote: post #4
amunozo wrote: post #2

Babe wake up. New benchmark to overfit models just dropped.

They’re definitely going to overfit on this, but this will be much better from a marketing perspective. Normies don’t know wtf an MMLU is, but they do know what IQ is and that 140 is a big number. Can’t wait for CEOs to start saying “why would we hire a 120 IQ person who works 9-5 with a lunch break when we can hire a 170 IQ worker who works 24x7 for half the cost??”

I sure hope they do, will finally give people some sense of just how little IQ tests actually say.

Which mirrors my general experience with automation and AI. Once something is automated and/or AI can do it, the magic goes away, and we're that much ahead in peeling back what is it that actually differentiates us and how exactly.

Re: IQ tests results for AI

#272

Post by WanderPanda »

Retric wrote: post #189
technothrasher wrote: post #117

Earlier quoted context omitted.

> The way human IQ testing developed is that researchers noticed people who excel in one cognitive task tend to do well in others My son took an IQ test and it wouldn't score him because he breaks this assumption. He was getting 98% in some tasks and 2% in others. The psychologist giving him the test said it was unlikely enough pattern that they couldn't get an IQ result for him. He's been diagnosed with non-verbal l…

IMO g is purely an abstraction. As long as the rate you learn most things is within a reasonable bound spending more or less time learning/perfecting X impacts the time you spend on Y, resulting in people being generally more or less proficient in a huge range of common cognitive skills. Thus, testing those general skills is normally a proxy for a wide range of things. LD breaks IQ because it results in noticeably un…

For me it all made sense when I heard that IQ/g-factor basically vanishes in the absence of time pressure (heard if from Richard Haier on Lex).

For a very narrow range of professions, like ATCs, time is absolutely critical but for most it does not really matter that much. Especially in many STEM fields. I think people in a broad IQ range can build abstractions and acquire intuitions about pretty complex matter. From this view-point ability to concentrate for long times, curiosity etc. seem more important than "raw-compute".

"if you value intelligence above all other human qualities, you’re gonna have a bad time" - Ilya

Timeless statement imo, even in the absence of AI

Re: IQ tests results for AI

#273

Post by machomaster »

robwwilliams wrote: post #179
bena wrote: post #142

Earlier quoted context omitted.

Don’t confuse knowledge with intelligence. Knowing what a trireme isn’t a sign of intelligence, it’s trivia. Being able to figure out how to build a trireme is intelligence. And I don’t mean looking up instructions and doing, I mean being able to reconstruct the knowledge from baser principles. This is not to say you or your son aren’t intelligent, just that knowing about triremes isn’t a sign of it.

Just a funny story. Exposure to vocabulary is an important factor in learning to think.

Has nothing to do with IQ. Knowing fancy words can only fool some people to think that the speaker is smarter; but it won't influence the actual IQ.

Re: IQ tests results for AI

#274

Post by BriggyDwiggs42 »

d4rkn0d3z wrote: post #266
sejje wrote: post #204

Earlier quoted context omitted.

How do your online scores contribute to the point?

Their difference is about the same. If you think this is a brag you must have also missed me saying the scores are meaningless in my view. The questions are great fun though. A better way to get at intelligence metrics would be to test over a number of years in many ways, again for humans.

The accuracy of the tests is known to drop above the ~140 range. Not a fan of IQ, but not convinced this means anything either.

Re: IQ tests results for AI

#275

Post by EnPissant »

robwwilliams wrote: post #156
alphazard wrote: post #70

Earlier quoted context omitted.

This is misleading. Anyone who wants to learn about IQ should Google it. It's the most replicated finding in psychology, and any questions you have about twins or groups with similar or different genes have probably been investigated. There is a lot of noise online in the form of commentary about IQ, so it's important to look at actual data if you are skeptical/curious.

Not sure what you mean precisely. Yes lots if work in IQ, but that does not mean there is a grand consensus. I am a geneticist who studies cognitive function. The single most common misunderstanding about estimates of heritability is that a high heritability implies full genetic causation without potential malleability. That is total wrong. Heritability is always measured in the context of Environment X. If you chang…

> Heritability is always measured in the context of Environment X. If you change to Environment Y or Z then the heritability will often change greatly.

That's not a very meaningful statement. If you took two twins and severely malnourished one of them it would not be useful to say: "See! IQ is mostly environmental!".

You have to assume some kind of baseline environment that nearly everyone will share, and that can be full-filled just by the virtue of growing up in a country like America. Otherwise, you are just concerning yourself with insignificant outliers.

Here is a twin study that places the heritability at ~80%: https://www.cambridge.org/core/journals/twin-research-and-hu...

Re: IQ tests results for AI

#276

Post by BriggyDwiggs42 »

block_dagger wrote: post #40
Telemakhos wrote: post #25

AI has a 140 “IQ” but understands nothing. That’s because AI does not understand anything: it just predicts the next token based on previous tokens and statistics. AI can give me five synonyms for any Latin word, because that’s just statistics, and it can regurgitate rules about metrical length of syllables, but it can’t give me synonyms matching a particular metrical pattern, because that would involve applying know…

An AI might say: a human with an IQ of 120 has an illusion of comprehension they call "understanding." It is an illusion because when you ask them to solve simple problems about a domain they claim to be masters of, they will take days or weeks to solve the problems that I can solve in minutes and the quality of their results will be lower than mine. Humans should reconsider what the meaning of learning and comprehen…

The phrasing here seems to almost imply consciousness isn’t real? It’s an incredibly hard problem that we only attempt to solve because the fact of consciousness is so undeniable.

Re: IQ tests results for AI

#277

Post by moritonal »

technothrasher wrote: post #258
moritonal wrote: post #202

Earlier quoted context omitted.

Just to add an anecdote, as of 15 years ago I had similar scores and was diagnosed with dyslexia.

My son struggled quite a bit learning to read. He was very slow to do so, and it frustrated him quite a lot. But interestingly, once he did get it, he became a voracious reader, and he's never since scored below 95th percentile on reading tests. So his developmental dyslexia did not carry over into general dyslexia. One of the interesting things about nvld, at least in his case, is that you would never know he had a…

To add a bit more, I didn't really read until later, around 8, but then read at a fairly quick and passionate amount for my age. People thought I'd dodged my family curse but later around 10 it was shown I was quite behind at school. My dyslexia was described as "a defect in one aspect of intelligence", which for most kids is reading comprehension, but for me it showed in a lack of decent short term memory. As you say, learning disabilities are interesting issues to deal with, but I'm sure your son will do great with support like you around him.

Re: IQ tests results for AI

#278

Post by eldenring »

gpt5 wrote: post #6

The way human IQ testing developed is that researchers noticed people who excel in one cognitive task tend to do well in others - the “positive manifold.” They then hypothesized a general factor, “g,” to explain this pattern. Early tests (e.g., Binet–Simon; later Stanford–Binet and Wechsler) sampled a wide range of tasks, and researchers used correlations and factor analysis to extract the common component, then norm…

I don't think something like this works when you can change the model, retrain it, etc. Or at least its much more difficult to do.

Re: IQ tests results for AI

#279

Post by marstall »

technothrasher wrote: post #117
gpt5 wrote: post #6

The way human IQ testing developed is that researchers noticed people who excel in one cognitive task tend to do well in others - the “positive manifold.” They then hypothesized a general factor, “g,” to explain this pattern. Early tests (e.g., Binet–Simon; later Stanford–Binet and Wechsler) sampled a wide range of tasks, and researchers used correlations and factor analysis to extract the common component, then norm…

> The way human IQ testing developed is that researchers noticed people who excel in one cognitive task tend to do well in others My son took an IQ test and it wouldn't score him because he breaks this assumption. He was getting 98% in some tasks and 2% in others. The psychologist giving him the test said it was unlikely enough pattern that they couldn't get an IQ result for him. He's been diagnosed with non-verbal l…

my son had similar results! we thought NVLD - now we are pretty certain Kabuki Syndrome.

Re: IQ tests results for AI

#280

Post by ahmedhawas123 »

jedberg wrote: post #228
ahmedhawas123 wrote: post #210

Would be great to add a few human benchmarks on this (e.g., average US IQ, Ivy League average, human 80th percentile). Also understanding some IQ per cost metric could be fun. Overall this is fun but not sure anyone in their right mind will be selecting an LLM based on this IQ benchmark

The IQ curve is designed so that 100 is average. So two of your questions can be answered with math: > average US IQ 100 >human 80th percentile 113 The last one, Ivy League average, can be guessed based on published data. The median SAT score of an Ivy League attendee is 1500. A 1500 on the SAT is roughly an IQ of 130-140. So in theory, the median Ivy League attendee is at the genius level.

Thanks. My comment was less a question of what the numbers are, but rather provide visual benchmarks on the IQ visualization, since IQ is a metric usually used for humans.
Post reply on HN