i guess inference engineering, like dpsark or dflash specific speculative decoding technqiues
AI is 'not smart' so what's next in artificial intelligence?
11–20 of 80 posts
Re: AI is 'not smart' so what's next in artificial intelligence?
#12AI winter.
Not likely. Take with whatever grain of salt you'd like, but that was largely a property of development being academicized and subject to things like grant cycles, research topic fashionability trends, and institutional structure. It would be wrong to assume it's some baked in thing that's guaranteed to happen independent of how development looks.
But _AI today_ is heavily subsidized by investor capital in the same way investors subsidized social/mobile/big data/VR/blockchain in the past. It's not unlikely "AI" would get a soft taboo in the same way as if you just presented a mobile-first, big-data driven, VR social media app today.
Which, judging by the terrible PR optics AI has nowadays, could unfortunately seep into academia too. Fund grants wouldn't want their names associated with anything with "AI" in its name even if it's a return to expert systems.
Re: AI is 'not smart' so what's next in artificial intelligence?
#13Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially.
and then models learned that they can back track and error correct
so much for "mathematically impossible..."
Re: AI is 'not smart' so what's next in artificial intelligence?
#14Ha before reading the article I thought "this must be an interview of Lecun". A bitter scientist that hates he was left behind the revolution.
Left behind how? It's been transformers since 2016 and not much actual progress in basic architectures has happened 10 years later. I'm honestly struggling to see how you can be left behind in this field.
and CPUs have the same basic architecture since 2000. no progress happened, right?
Re: AI is 'not smart' so what's next in artificial intelligence?
#15It depends on how you define "smart".
For me, "smart" means doing things less based on instinct. Things humans can do but mice cannot, things mathematicians can do but normal people cannot, etc.
Considering the unit distance conjecture was disproved by OAI's model last month, I think maybe LLMs should count as "smart".
Re: AI is 'not smart' so what's next in artificial intelligence?
#16The article seems to define "smart" as being good at spatial awareness and navigating a body through 3D space and such. Thus, a mice is smarter than an LLM. That's the first time in my life I hear this definition. Until now, the word "smart" has meant doing exactly the things LLMs do, and mice don't. I guess it is a sign we are re-evaluating what makes humans special.
> I guess it is a sign we are re-evaluating what makes humans special.
Always has been: https://en.wikipedia.org/wiki/AI_effect
Tangentially: https://en.wikipedia.org/wiki/Moravec%27s_paradox
Re: AI is 'not smart' so what's next in artificial intelligence?
#17Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."
I think it was largely the introduction of tool calling that allowed models to mitigate the issue of errors amplifying exponentially since it allows the model to understand if what it generated is correct or has issues that it needs to address. This addresses the potential lack of or low quality of world model by being able to reference the current state of the world.
Re: AI is 'not smart' so what's next in artificial intelligence?
#18Besides "smart", the headline also conflates AI with LLMs. The real, non-clickbait title is "Yann LeCun, founder of AMI Labs, is developing a new AI system"
Re: AI is 'not smart' so what's next in artificial intelligence?
#19Yann LeCun was saying 3 years ago that because token generation is auto-regressive, its mathematically impossible to generate a long stream of coherent tokens, because errors amplify exponentially. and then models learned that they can back track and error correct so much for "mathematically impossible..."
Also, almost any argument against LLM intelligence also applies to humans.
I very commonly see someone make some small mistake and end up going in the wrong direction, “accumulating stupid” as they go, sometimes for years.
Re: AI is 'not smart' so what's next in artificial intelligence?
#20Its definitely not as dumb as MAGA crowd