Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI capability isn't humanness

research.roundtable.ai

Post reply on HN
Print view View original

11–20 of 55 posts

Re: AI capability isn't humanness

#11

Post by ACCount37 »

cortesoft wrote: post #9
ForceBru wrote: post #5

> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given mod…

Not only that, but humans also have access to all of the "training data" of hundreds of millions of years of evolution baked into our brains.

Which must be doing some heavy lifting.

Humans ship with all the priors evolution has managed to cram into them. LLMs have to rediscover all of it from scratch just by looking at an awful lot of data.

Re: AI capability isn't humanness

#12

Post by ACCount37 »

gmuslera wrote: post #10

LLMs are language models. We interact with them using language, all of that, but also only that. That doesn't mean that they have "common sense", context, same motivations, agency, or even reasoning like us. But as we interact with other people using mostly language, and since the start of internet a lot of those interactions happen in way similar to how we interact with AI, the difference is not so obvious. We are f…

"Language" is just the interface. What happens on the inside of LLMs is a lot weirder than that.

Re: AI capability isn't humanness

#13

Post by emp17344 »

ForceBru wrote: post #5

> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given mod…

It’s unlikely sensory data contributes to intelligence in human beings. Blind people take in far, far less sensory data than sighted people, and yet are no less intelligent. Think of Helen Keller - she was deafblind from an early age, and yet was far more intelligent than the average person. If your hypothesis is correct, and development of human intelligence is primarily driven by sensory data, how do you reconcile this with our observations of people with sensory impairments?

Re: AI capability isn't humanness

#14

Post by jakeinspace »

emp17344 wrote: post #13
ForceBru wrote: post #5

> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given mod…

It’s unlikely sensory data contributes to intelligence in human beings. Blind people take in far, far less sensory data than sighted people, and yet are no less intelligent. Think of Helen Keller - she was deafblind from an early age, and yet was far more intelligent than the average person. If your hypothesis is correct, and development of human intelligence is primarily driven by sensory data, how do you reconcile…

Blind people tend to have less spatial intelligence though, like significantly more. Not very nice to say like that, and of course they often develop heightened intelligence in other areas, but we do consider human-level spatial reasoning a very important goal in AI.

Re: AI capability isn't humanness

#15

Post by moffkalast »

ForceBru wrote: post #5

> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given mod…

There's only so much information content you can get from a mug though.

We get a lot of high quality data that's relatively the same. We run the same routines every day, doing more or less the same things, which makes us extremely reliable at what we do but not very worldly.

LLMs get the opposite: sparse, relatively low quality, low modality data that's extremely varied, so they have a much wider breadth of knowledge but they're pretty fragile in comparison since they get relatively little experience on each topic and usually no chance to affirm learning with RL.

Re: AI capability isn't humanness

#16

Post by lawlessone »

ACCount37 wrote: post #12
gmuslera wrote: post #10

LLMs are language models. We interact with them using language, all of that, but also only that. That doesn't mean that they have "common sense", context, same motivations, agency, or even reasoning like us. But as we interact with other people using mostly language, and since the start of internet a lot of those interactions happen in way similar to how we interact with AI, the difference is not so obvious. We are f…

"Language" is just the interface. What happens on the inside of LLMs is a lot weirder than that.

I feel like the interface in this case has caused us to fool ourselves into thinking there's more there than there is.

Before 2022 (most of history), if you had a long seemingly sensible conversation with something, you could safely assume this other party was a real thinking human mind.

it's like a duck call.

edit, i want to add because this is neural net that's trained to output sensible text, language isn't just the interface.

unlike a website there's no separation between anything, with LLM's the back and front end are all one blob.

edit2: seems I have upset the ducks that think the duck call is a real duck .

Re: AI capability isn't humanness

#17

Post by layer8 »

cortesoft wrote: post #9
ForceBru wrote: post #5

> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given mod…

Not only that, but humans also have access to all of the "training data" of hundreds of millions of years of evolution baked into our brains.

I don’t think the amount of data is essential here. The human genome is only around 750 MB, much less than current LLMs, and likely only a small fraction of it determines human intelligence. On the other hand, current LLMs contain immense amounts of factual knowledge that a human newborn carries zero information about.

Intelligence likely doesn’t require that much data, and it may be more a question of evolutionary chance. After all, human intelligence is largely (if not exclusively) the result of natural selection from random mutations, with a generation count that’s likely smaller than the number of training iterations of LLMs. We haven’t found a way yet to artificially develop a digital equivalent effectively, and the way we are training neural networks might actually be a dead end here.

Re: AI capability isn't humanness

#18

Post by dpark »

emp17344 wrote: post #13
ForceBru wrote: post #5

> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given mod…

It’s unlikely sensory data contributes to intelligence in human beings. Blind people take in far, far less sensory data than sighted people, and yet are no less intelligent. Think of Helen Keller - she was deafblind from an early age, and yet was far more intelligent than the average person. If your hypothesis is correct, and development of human intelligence is primarily driven by sensory data, how do you reconcile…

> It’s unlikely sensory data contributes to intelligence in human beings.

This is clearly untrue. All information a human ever receives is through sensory data. Unless your position is that the intelligence of a brain that was grown in a vat with no inputs would be equivalent to that of a normal person.

Now, does rotating a coffee mug and feeling its weight, seeing it from different angles, etc. improve intelligence? Actually, still yes, if your intelligence test happens to include questions like “is this a picture of a mug” or “which of these objects is closest in weight to a mug”.

Re: AI capability isn't humanness

#19

Post by freejazz »

ACCount37 wrote: post #12
gmuslera wrote: post #10

LLMs are language models. We interact with them using language, all of that, but also only that. That doesn't mean that they have "common sense", context, same motivations, agency, or even reasoning like us. But as we interact with other people using mostly language, and since the start of internet a lot of those interactions happen in way similar to how we interact with AI, the difference is not so obvious. We are f…

"Language" is just the interface. What happens on the inside of LLMs is a lot weirder than that.

And?

Re: AI capability isn't humanness

#20

Post by emp17344 »

jakeinspace wrote: post #14
emp17344 wrote: post #13

Earlier quoted context omitted.

It’s unlikely sensory data contributes to intelligence in human beings. Blind people take in far, far less sensory data than sighted people, and yet are no less intelligent. Think of Helen Keller - she was deafblind from an early age, and yet was far more intelligent than the average person. If your hypothesis is correct, and development of human intelligence is primarily driven by sensory data, how do you reconcile…

Blind people tend to have less spatial intelligence though, like significantly more. Not very nice to say like that, and of course they often develop heightened intelligence in other areas, but we do consider human-level spatial reasoning a very important goal in AI.

People with sensory impairments from birth may be restricted in certain areas, on account of the sensory impairment, but are no less generally cognitively capable than the average person.
Post reply on HN