Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spatial intelligence is AI’s next frontier

drfeifei.substack.com

Post reply on HN
Print view View original

111–120 of 136 posts

Re: Spatial intelligence is AI’s next frontier

#111

Post by nycdatasci »

nosianu wrote: post #101
Animats wrote: post #80

Earlier quoted context omitted.

> From reading that, I'm not quite sure if they have anything figured out. I actually agree, but her notes are mostly fluff with no real info in there and I do wonder if they have anything figured out besides "collect spatial data" like imagenet. Right. I was thinking about this back in the 1990s. That resulted in a years-long detour through collision detection, physically based animation, solving stiff systems of no…

> I used to make the comment, pre-LLM, that we needed to get to mouse/squirrel level intelligence rather than trying to get to human level abstract AI. But we got abstract AI first. That surprised me. "AI" is not based on physical real world data and models like our brain. Instead, we chose to analyze human formal (written) communication. ("formal": actual face to face communication has tons of dimensions adding to t…

You seem to be suggesting that current frontier models are only trained on text and not "sensor data". Multi-modal models are trained on the entire internet + vast amounts of synthetic data. Images and videos are key inputs. Camera sensors are capable of capturing much more "sensor data" than the human eye. Neural networks are the worst way to model intelligence, except all other models.

You may find this talk enlightening: https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023...

Re: Spatial intelligence is AI’s next frontier

#112

Post by imtringued »

Animats wrote: post #80
toisanji wrote: post #13

From reading that, I'm not quite sure if they have anything figured out. I actually agree, but her notes are mostly fluff with no real info in there and I do wonder if they have anything figured out besides "collect spatial data" like imagenet. There are actually a lot of people trying to figure out spatial intelligence, but those groups are usually in neuroscience or computational neuroscience. Here is a summary pap…

> From reading that, I'm not quite sure if they have anything figured out. I actually agree, but her notes are mostly fluff with no real info in there and I do wonder if they have anything figured out besides "collect spatial data" like imagenet. Right. I was thinking about this back in the 1990s. That resulted in a years-long detour through collision detection, physically based animation, solving stiff systems of no…

>There's some progress in video generation which takes a short clip and extrapolates what happens next. That's a promising line of development. The key to "common sense" is being able to predict what happens next well enough to avoid big mistakes in the short term, a few seconds. How's that coming along? And what's the internal world model, assuming we even know?

https://www.youtube.com/watch?v=udPY5rQVoW0

This has been a thing for a while. It's actually a funny way to demonstrate model based control by replacing the controller with a human.

Re: Spatial intelligence is AI’s next frontier

#113

Post by imtringued »

tim333 wrote: post #62
jacquesm wrote: post #46

I think I perceive a massive bottleneck. Today's incarnation of AI learns from the web, not from the interaction with the humans it talks to. And for sure there is a lot of value there, it is just pointless to see that interaction lost a few hundred or thousand words of context later. For humans their 'context' is their life and total memory capacity, that's why we learn from the interaction with other, more experien…

There was stuff on a possible way around that from Google Research out the other day called Nested Learning https://research.google/blog/introducing-nested-learning-a-n... My understanding is at the moment you train something like ChatGPT on the web, setting weights with backpropagation till it works well, but if you give some more info and do more backprop it can forget other stuff it's learned, called 'catastrophic…

That's pretty cool that you point it out.

>We introduce Nested Learning, a new approach to machine learning that views models as a set of smaller, nested optimization problems, each with its own internal workflow, in order to mitigate or even completely avoid the issue of “catastrophic forgetting”, where learning new tasks sacrifices proficiency on old tasks. [0].

It feels funny to be vindicated by rambling something random a week before someone makes an announcement that they did something incredibly similar with great success:

>Here is my stupid and simple unproven idea: Nest the reinforcement learning algorithm. Each critic will add one more level of delay, thereby acting as a low pass filter on the supervised reward function. Since you have two critics now, you can essentially implement a hybrid pre-training + continual learning architecture. The most interesting aspect here is that you can continue training the inner-most critic without changing the outer critic, which now acts as a learned loss function. [1]

[0] https://research.google/blog/introducing-nested-learning-a-n... [1] https://news.ycombinator.com/item?id=45745402

Re: Spatial intelligence is AI’s next frontier

#115

Post by Marshferm »

toisanji wrote: post #13

From reading that, I'm not quite sure if they have anything figured out. I actually agree, but her notes are mostly fluff with no real info in there and I do wonder if they have anything figured out besides "collect spatial data" like imagenet. There are actually a lot of people trying to figure out spatial intelligence, but those groups are usually in neuroscience or computational neuroscience. Here is a summary pap…

To decipher if there is anything like spatial intelligence, which is an oxymoronic term at most and redundant at least, one has to decipher the base units of the processes prior to their materialization in the allocortex. And to assign a careful concatenated/parametric categorization of what is unitized, where the processes focus into thresholds etc. This frontier propaganda and the few arvix/nature papers here are too synthetic to lead anywhere of merit.

Re: Spatial intelligence is AI’s next frontier

#116

Post by Marshferm »

ACCount37 wrote: post #107
toisanji wrote: post #13

From reading that, I'm not quite sure if they have anything figured out. I actually agree, but her notes are mostly fluff with no real info in there and I do wonder if they have anything figured out besides "collect spatial data" like imagenet. There are actually a lot of people trying to figure out spatial intelligence, but those groups are usually in neuroscience or computational neuroscience. Here is a summary pap…

The question, as always, is: can we get any useful insights from all of that? Trying to copy biological systems 1:1 rarely works, and copying biological systems doesn't seem to be required either. CNNs are somewhat brain-inspired, but only somewhat, and LLMs have very little architectural similarity to human brain - other than being an artificial neural network. This functional similarity of LLMs to the human brain d…

There's nothing similar about LLMs and human brains. Theyre entirely divergent. Training a machine has nothing remotely to do with biological development.

Re: Spatial intelligence is AI’s next frontier

#117

Post by jacquesm »

imtringued wrote: post #113
tim333 wrote: post #62

Earlier quoted context omitted.

There was stuff on a possible way around that from Google Research out the other day called Nested Learning https://research.google/blog/introducing-nested-learning-a-n... My understanding is at the moment you train something like ChatGPT on the web, setting weights with backpropagation till it works well, but if you give some more info and do more backprop it can forget other stuff it's learned, called 'catastrophic…

That's pretty cool that you point it out. >We introduce Nested Learning, a new approach to machine learning that views models as a set of smaller, nested optimization problems, each with its own internal workflow, in order to mitigate or even completely avoid the issue of “catastrophic forgetting”, where learning new tasks sacrifices proficiency on old tasks. [0]. It feels funny to be vindicated by rambling something…

That's a pretty amazing line-up of events.

Re: Spatial intelligence is AI’s next frontier

#118

Post by ACCount37 »

Marshferm wrote: post #116
ACCount37 wrote: post #107

Earlier quoted context omitted.

The question, as always, is: can we get any useful insights from all of that? Trying to copy biological systems 1:1 rarely works, and copying biological systems doesn't seem to be required either. CNNs are somewhat brain-inspired, but only somewhat, and LLMs have very little architectural similarity to human brain - other than being an artificial neural network. This functional similarity of LLMs to the human brain d…

There's nothing similar about LLMs and human brains. Theyre entirely divergent. Training a machine has nothing remotely to do with biological development.

They perform incredibly similar functions. Thus, "functionally similar".

Re: Spatial intelligence is AI’s next frontier

#119

Post by Marshferm »

ACCount37 wrote: post #118
Marshferm wrote: post #116

Earlier quoted context omitted.

There's nothing similar about LLMs and human brains. Theyre entirely divergent. Training a machine has nothing remotely to do with biological development.

They perform incredibly similar functions. Thus, "functionally similar".

There’s no functional similarity in the slightest. Notice you can’t cite examples.

Re: Spatial intelligence is AI’s next frontier

#120

Post by ACCount37 »

Marshferm wrote: post #119
ACCount37 wrote: post #118

Earlier quoted context omitted.

They perform incredibly similar functions. Thus, "functionally similar".

There’s no functional similarity in the slightest. Notice you can’t cite examples.

Hard metrics: LLMs perform NLP, NLU and CSR tasks at humanlike levels.

Research findings: LLMs have and use world models. They use some type of abstract thinking - with internal representation that often correspond to human abstract concepts. Which adds up to a capability profile that's amusingly humanlike.

Humans, however, don't like that. They really don't. AI effect is too strong, and it demands that humans must be Special. So some humans, when faced with the possibility that an AI might be doing the same thing their own brains do, resort to coping and seething.

Post reply on HN