Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is 'not smart' so what's next in artificial intelligence?

bbc.com

Post reply on HN
Print view View original

41–50 of 80 posts

Re: AI is 'not smart' so what's next in artificial intelligence?

#41

Post by rcxdude »

TMWNN wrote: post #25
nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

> and then models learned that they can back track and error correct You mean "Human developers learned ...", yes? Or was there really an all AI-driven, self-improving aspect to this?

Well, LLM networks don't have a 'back track and error correct' component in the design, AFAIK.

Re: AI is 'not smart' so what's next in artificial intelligence?

#42

Post by threethirtytwo »

shevy-java wrote: post #39
jiggawatts wrote: post #19

Earlier quoted context omitted.

Also, almost any argument against LLM intelligence also applies to humans. I very commonly see someone make some small mistake and end up going in the wrong direction, “accumulating stupid” as they go, sometimes for years.

Humans can learn. AI can not. For those disagreeing: please explain how a static hardware can learn.

this is profoundly false. AI not only can learn, it is built entirely from learning. The field is called machine learning after all.

Not only that... AI is NOT only learning during the training phase... LLMs learn in real time the minute you talk to it. It learns something and saves those learnings in a context window or somewhere else if you want it to exist beyond the context window.

All of the above runs on static hardware. Don't understand how someone can say a profoundly wrong statement and get voted up.

Re: AI is 'not smart' so what's next in artificial intelligence?

#43

Post by altmanaltman »

armchairhacker wrote: post #18

Besides "smart", the headline also conflates AI with LLMs. The real, non-clickbait title is "Yann LeCun, founder of AMI Labs, is developing a new AI system"

They literally interview another person in it and mention a lot of other labs doing this kind of research including Google. Yes, he's the main starting interview but this is not really clickbait or a marketing piece.

Re: AI is 'not smart' so what's next in artificial intelligence?

#44

Post by someonebaggy »

shevy-java wrote: post #39
jiggawatts wrote: post #19

Earlier quoted context omitted.

Also, almost any argument against LLM intelligence also applies to humans. I very commonly see someone make some small mistake and end up going in the wrong direction, “accumulating stupid” as they go, sometimes for years.

Humans can learn. AI can not. For those disagreeing: please explain how a static hardware can learn.

idk, how does voice recognition learn my voice? How can I install programs when the hardware is static?

Re: AI is 'not smart' so what's next in artificial intelligence?

#45

Post by karahime »

JsonDemWitOster wrote: post #32
karahime wrote: post #21

Earlier quoted context omitted.

You're mixing different things. Mobile first is integrated into new services to the point that they either are mobile first, or they have a design system which includes mobile as a surface. VR has a wide user base (MQ2 sold as well as the original Xbox) and is involved in both manufacturing design and simulation, and is hardly an academic taboo, even if the "main" topic of discussion is elsewhere right now. Blockchai…

Fair enough. Mobile and social became ubiquitous and are now table stakes. But my problem with VR and blockchain---even allowing for the fact/assumption that they are still relevant---is that they never lived up to their hype. They never became ubiquitous as mobile and social. They don't inspire investor confidence like they did in the past, if at all. AI, if it survives the public and regulatory backlash, could be h…

The vast consumer adoption and ongoing involvement seems to point the other way, though. I think a lot of the appearance of backlash is on (specifically anglophone, mostly) social media, which is going through a somewhat reactionary phase regardless.

Re: AI is 'not smart' so what's next in artificial intelligence?

#46

Post by rcxdude »

shevy-java wrote: post #38
nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

You insinuate here AI "learned". I doubt that this was AI self-improvement.

Was there a particular change to the network or the way that it was trained that introduced the 'backtrack and error correct' mechanism?

Re: AI is 'not smart' so what's next in artificial intelligence?

#47

Post by echoangle »

shevy-java wrote: post #39
jiggawatts wrote: post #19

Earlier quoted context omitted.

Also, almost any argument against LLM intelligence also applies to humans. I very commonly see someone make some small mistake and end up going in the wrong direction, “accumulating stupid” as they go, sometimes for years.

Humans can learn. AI can not. For those disagreeing: please explain how a static hardware can learn.

By self-modifying the software. Currently the model harnesses only allow the model to modify its own prompt (which could be considered a really weak kind of learning), but theoretically, a model could design and train its own replacement and run that, continuously improving itself. I’m not sure if LLMs will be able to do that but the static hardware has nothing to do with it (since the bits on the harddrive aren’t static).

Re: AI is 'not smart' so what's next in artificial intelligence?

#48

Post by threethirtytwo »

nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

Stop attacking Yann. I would say like 90% of the HN crowd was parroting Yann too.

Re: AI is 'not smart' so what's next in artificial intelligence?

#49

Post by infinite_spin »

shevy-java wrote: post #38
nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

You insinuate here AI "learned". I doubt that this was AI self-improvement.

do you have a problem with this field of research being called "machine learning"?

Re: AI is 'not smart' so what's next in artificial intelligence?

#50

Post by guenthert »

threethirtytwo wrote: post #42
shevy-java wrote: post #39

Earlier quoted context omitted.

Humans can learn. AI can not. For those disagreeing: please explain how a static hardware can learn.

this is profoundly false. AI not only can learn, it is built entirely from learning. The field is called machine learning after all. Not only that... AI is NOT only learning during the training phase... LLMs learn in real time the minute you talk to it. It learns something and saves those learnings in a context window or somewhere else if you want it to exist beyond the context window. All of the above runs on static…

Correct me if I'm wrong, but if a profound insight is gathered in session 1 with user A and stored in context A1, this might be available to user A in session 2, if that still has access to context A1, but won't be available to user B in any of his/her sessions until that NN is retrained with input which includes at least some of the information from context A1.
Post reply on HN