Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is 'not smart' so what's next in artificial intelligence?

bbc.com

Post reply on HN
Print view View original

21–30 of 80 posts

Re: AI is 'not smart' so what's next in artificial intelligence?

#21

Post by karahime »

JsonDemWitOster wrote: post #12
karahime wrote: post #8

Earlier quoted context omitted.

Not likely. Take with whatever grain of salt you'd like, but that was largely a property of development being academicized and subject to things like grant cycles, research topic fashionability trends, and institutional structure. It would be wrong to assume it's some baked in thing that's guaranteed to happen independent of how development looks.

But _AI today_ is heavily subsidized by investor capital in the same way investors subsidized social/mobile/big data/VR/blockchain in the past. It's not unlikely "AI" would get a soft taboo in the same way as if you just presented a mobile-first, big-data driven, VR social media app today. Which, judging by the terrible PR optics AI has nowadays, could unfortunately seep into academia too. Fund grants wouldn't want t…

You're mixing different things. Mobile first is integrated into new services to the point that they either are mobile first, or they have a design system which includes mobile as a surface. VR has a wide user base (MQ2 sold as well as the original Xbox) and is involved in both manufacturing design and simulation, and is hardly an academic taboo, even if the "main" topic of discussion is elsewhere right now. Blockchains are being integrated into financial infrastructure even as some people make snarky commentary about it. Sometimes optics is just an optical illusion.

Re: AI is 'not smart' so what's next in artificial intelligence?

#22

Post by waldarbeiter »

nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

[dead]

Re: AI is 'not smart' so what's next in artificial intelligence?

#23

Post by SwellJoe »

feverzsj wrote: post #4

AI winter.

We're past the point where there's a feasible argument that there is an AI winter coming.

The models work remarkably well for several classes of problem that seemed impossible a few years ago. They're not going away. There will still absolutely be a lot of ups and down and crazy stuff that happens in AI, but it won't be that AI almost completely stops being developed/funded for a decade or more. The biggest risk, I think, is regulatory capture; it's what Anthropic and OpenAI seem to be aiming for with their scaremongering about how capable and dangerous their models are. That'll put a damper on the industry for everyone except the companies that bribe the right people.

Re: AI is 'not smart' so what's next in artificial intelligence?

#25

Post by TMWNN »

nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

> and then models learned that they can back track and error correct

You mean "Human developers learned ...", yes? Or was there really an all AI-driven, self-improving aspect to this?

Re: AI is 'not smart' so what's next in artificial intelligence?

#26

Post by cauch »

JsonDemWitOster wrote: post #16
dagss wrote: post #10

The article seems to define "smart" as being good at spatial awareness and navigating a body through 3D space and such. Thus, a mice is smarter than an LLM. That's the first time in my life I hear this definition. Until now, the word "smart" has meant doing exactly the things LLMs do, and mice don't. I guess it is a sign we are re-evaluating what makes humans special.

> I guess it is a sign we are re-evaluating what makes humans special. Always has been: https://en.wikipedia.org/wiki/AI_effect Tangentially: https://en.wikipedia.org/wiki/Moravec%27s_paradox

While we should be careful of a bias, it is also a good practice in the scientific method to review definitions that may have been not precise enough.

For example, initially, a "planet" was just a big body in space. Then when people started to see more and more nuances, the definition just refined, and some objects stopped being called "planet".

I would not be surprised if there is a bias that pushes some people to redefine "intelligence" away from machine, but I would not be surprised if there is a bias that pushes some people to ignore newly discovered nuance and put into the same "intelligence" bag things that are in fact very different. I personally can see how LLM are not really "intelligent", and I don't think it is a good idea to say: well, yesterday we said the minimum criteria is X, now that we noticed that X can be reached without really doing the real thing, let's just ignore that and pretend it is the same thing.

(: the biggest clue for me is to use an early model, and see that it sometimes looks very intelligent, and then sometimes you can see that it gets it wrong in a way that shows that it never "understood" it at all. Newer models are better, but because it is an iteration on the same bases, the increase of performances cannot really due to replacing the things that "looked smart by aren't" by "real smart", but more replacing the things that "don't look smart" by "look smart by aren't")

Re: AI is 'not smart' so what's next in artificial intelligence?

#27

Post by randsorex »

armchairhacker wrote: post #18

Besides "smart", the headline also conflates AI with LLMs. The real, non-clickbait title is "Yann LeCun, founder of AMI Labs, is developing a new AI system"

It is just so bizarre compared to my everyday experience also.

I never ask Opus or Fable a question and think "what a stupid response".

Quite the opposite. It has actually raised the bar of what I consider an intelligent response to my inquiry. So much so that most responses from humans on most subjects to most forms of inquiry seem stupid and not really thought out.

Re: AI is 'not smart' so what's next in artificial intelligence?

#28

Post by ravenstine »

charcircuit wrote: post #17
nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

I think it was largely the introduction of tool calling that allowed models to mitigate the issue of errors amplifying exponentially since it allows the model to understand if what it generated is correct or has issues that it needs to address. This addresses the potential lack of or low quality of world model by being able to reference the current state of the world.

I've definitely realized this phenomenon after a few occasions of erroneously trying to rely purely on instructions to get an LLM to do a thing or take on a role, especially without persistent cloud-based sessions that have internal checklists and other opaque guidance. They're essentially poor at self-managing, but can do really well when they are limited in scope/context and are worked into a sort of state machine that guarantees they perform certain tasks predictably. They won't always do those tasks the exact way you expect them to, but at least they actually do them, and because of that they are more likely to have the correct prior context to perform the next task better. Because they are so prone to selectively ignoring directions, that can quickly send them down an incorrect path that compounds on itself.

Re: AI is 'not smart' so what's next in artificial intelligence?

#29

Post by menaerus »

imtringued wrote: post #9
arisAlexis wrote: post #5

Ha before reading the article I thought "this must be an interview of Lecun". A bitter scientist that hates he was left behind the revolution.

Left behind how? It's been transformers since 2016 and not much actual progress in basic architectures has happened 10 years later. I'm honestly struggling to see how you can be left behind in this field.

Obviously, transformers architecture is just one of the ingredients. Otherwise we wouldn't be seeing competing labs in this race. I also read all his interviews as a marketing material.

Re: AI is 'not smart' so what's next in artificial intelligence?

#30

Post by fragmede »

jiggawatts wrote: post #19
nok22kon wrote: post #13

Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."

Also, almost any argument against LLM intelligence also applies to humans. I very commonly see someone make some small mistake and end up going in the wrong direction, “accumulating stupid” as they go, sometimes for years.

Also with the stochastic parrot thing. If you say just the right thing to the right human and the right time, they'll very predictibly say their favorite movie/book quote or song lyric, like some sort of parrot.
Post reply on HN