Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spatial intelligence is AI’s next frontier

drfeifei.substack.com

Post reply on HN
Print view View original

61–70 of 136 posts

Re: Spatial intelligence is AI’s next frontier

#61

Post by lsllc »

inshard wrote: post #42

Also good context here is Friston’s Free Energy Principle: A unified theory suggesting that all living systems, from simple organisms to the brain, must minimize "surprise" to maintain their form and survive. To do this, systems act to minimize a mathematical quantity called variational free energy, which is an upper bound on surprise. This involves constantly making predictions about the world, updating internal mod…

> taking actions that reduce the difference between predictions and reality, effectively minimizing prediction errors

Since you can't change reality itself, and you can only take actions to reduce variational free energy, doesn't this make everything into a self-fulfilling prophecy?

I guess there must be some base level of instinct that overrides this; in the case of "I think that sabertooth tiger is going to eat me" you want to make sure the "don't get eaten" instinct counters "minimizing prediction errors".

Re: Spatial intelligence is AI’s next frontier

#62

Post by tim333 »

jacquesm wrote: post #46

I think I perceive a massive bottleneck. Today's incarnation of AI learns from the web, not from the interaction with the humans it talks to. And for sure there is a lot of value there, it is just pointless to see that interaction lost a few hundred or thousand words of context later. For humans their 'context' is their life and total memory capacity, that's why we learn from the interaction with other, more experien…

There was stuff on a possible way around that from Google Research out the other day called Nested Learning https://research.google/blog/introducing-nested-learning-a-n...

My understanding is at the moment you train something like ChatGPT on the web, setting weights with backpropagation till it works well, but if you give some more info and do more backprop it can forget other stuff it's learned, called 'catastrophic forgetting'. The nested learning approach is to split things into a number of smaller models so you can retrain one without mucking up the other ones.

Re: Spatial intelligence is AI’s next frontier

#63

Post by alexose »

inciampati wrote: post #52

Just had a fantastic experience applying agentic coding to CAD. I needed to add some threads to a few blanks in a 3d print. I used computational geometry to give the agent a way to "feel" around the model. I had it convolve a sphere of the radius of the connector across the entire model. It was able to use this technique to find the precise positions of the existing ports and then add threads to them. It took a few t…

Generative CAD has incredible potential. I've had some decent results with OpenSCAD, but it's clear that current models don't have much "common sense" when it comes to how shapes connect.

If code-based CAD tools were more common, and we had a bigger corpus to pull from, these tools would probably be pretty usable. Without this, however, it seems like we'll need to train against simulations of the physical world.

Re: Spatial intelligence is AI’s next frontier

#64

Post by jgord »

My take, after working on some algos to detect geometry from pointclouds, is that its solvable with current ML techniques, but we lack early stage VC funding for startups working on this :

https://quantblog.wordpress.com/2025/10/29/digital-twins-the...

I have no doubt FeiFei and her well funded team will make rapid progress.

Re: Spatial intelligence is AI’s next frontier

#65

Post by halfcat »

jacquesm wrote: post #46

I think I perceive a massive bottleneck. Today's incarnation of AI learns from the web, not from the interaction with the humans it talks to. And for sure there is a lot of value there, it is just pointless to see that interaction lost a few hundred or thousand words of context later. For humans their 'context' is their life and total memory capacity, that's why we learn from the interaction with other, more experien…

> For humans their 'context' is their life and total memory capacity

And some number of billions of years of evolutionary progress.

Whatever spacial understanding we have could be thought of as a simulation at a quantum level, the size of the universe, for billions of years.

And what can we simulate completely at a quantum level today? Atoms or single cells?

Re: Spatial intelligence is AI’s next frontier

#66

Post by porphyra »

toisanji wrote: post #13

From reading that, I'm not quite sure if they have anything figured out. I actually agree, but her notes are mostly fluff with no real info in there and I do wonder if they have anything figured out besides "collect spatial data" like imagenet. There are actually a lot of people trying to figure out spatial intelligence, but those groups are usually in neuroscience or computational neuroscience. Here is a summary pap…

> if they have anything figured out besides "collect spatial data" like imagenet

I mean she launched her whole career with imagenet so you can hardly blame her for thinking that way. But on the other hand, there's something bitter lesson-pilled about letting a model "figure out" spatial relationships just by looking at tons of data. And tbh the recent progress [1] of worldlabs.ai (Dr Fei Fei Li's startup) looks quite promising for a model that understands stuff including reflections and stuff.

[1] https://www.worldlabs.ai/blog/rtfm

Re: Spatial intelligence is AI’s next frontier

#67

Post by john_minsk »

jgord wrote: post #64

My take, after working on some algos to detect geometry from pointclouds, is that its solvable with current ML techniques, but we lack early stage VC funding for startups working on this : https://quantblog.wordpress.com/2025/10/29/digital-twins-the... I have no doubt FeiFei and her well funded team will make rapid progress.

We think alike. Have you tried to replace point cloud of white wall with a generic white wall automatically?

Re: Spatial intelligence is AI’s next frontier

#68

Post by davmre »

byearthithatius wrote: post #31
yzydserd wrote: post #26

Earlier quoted context omitted.

> Fei-Fei: ??? Underrated and unsung. Fei Fei Li first launched ImageNet way back in 2007, a hugely influential move sparking much of the computer vision deep learning that followed since. I remember in a lecture about 7 years ago jph00 saying "text is just waiting for its imagenet moment" -> then came the gpt explosion. Fei Fei was massively instrumental in where we are today.

Curating a dataset is vastly different than introducing a new architectural approach. ImageNet is a database. Its not like inventing the convolutions for CNNs or the LSTM or a Transformer.

It's true that these are very different activities, but I think most ML researchers would agree that it's actually the creation of ImageNet that sparked the deep learning revolution. CNNs were not a novel method in 2012; the novelty was having a dataset big and sophisticated enough that it was actually possible to learn a good vision model from without needing to hand-engineer all the parts. Fei-fei saw this years in advance and invested a lot of time and career capital setting up the conditions for the bitter lesson to kick in. Building the dataset was 'easy' in a technical sense, but knowing that a big dataset was what the field needed, and staking her career on it when no one else was doing or valuing this kind of work, was her unique contribution, and took quite a bit of both insight and courage.

Re: Spatial intelligence is AI’s next frontier

#69

Post by godelski »

porphyra wrote: post #66
toisanji wrote: post #13

From reading that, I'm not quite sure if they have anything figured out. I actually agree, but her notes are mostly fluff with no real info in there and I do wonder if they have anything figured out besides "collect spatial data" like imagenet. There are actually a lot of people trying to figure out spatial intelligence, but those groups are usually in neuroscience or computational neuroscience. Here is a summary pap…

> if they have anything figured out besides "collect spatial data" like imagenet I mean she launched her whole career with imagenet so you can hardly blame her for thinking that way. But on the other hand, there's something bitter lesson-pilled about letting a model "figure out" spatial relationships just by looking at tons of data. And tbh the recent progress [1] of worldlabs.ai (Dr Fei Fei Li's startup) looks quite…

  > looks quite promising for a model that understands stuff including reflections and stuff.
I got the opposite impression when trying their demo...[0]. Even in their examples some of these issues exist like how objects stay a constant size despite moving. Like missing the parallax or depth information. Not to mention that they show it walking on water lol

As for reflections, I don't get that impression either. They seem extremely brittle to movement.

[0] http://0x0.st/K95T.png

Re: Spatial intelligence is AI’s next frontier

#70

Post by inshard »

lsllc wrote: post #61
inshard wrote: post #42

Also good context here is Friston’s Free Energy Principle: A unified theory suggesting that all living systems, from simple organisms to the brain, must minimize "surprise" to maintain their form and survive. To do this, systems act to minimize a mathematical quantity called variational free energy, which is an upper bound on surprise. This involves constantly making predictions about the world, updating internal mod…

> taking actions that reduce the difference between predictions and reality, effectively minimizing prediction errors Since you can't change reality itself, and you can only take actions to reduce variational free energy, doesn't this make everything into a self-fulfilling prophecy? I guess there must be some base level of instinct that overrides this; in the case of "I think that sabertooth tiger is going to eat me"…

Yep. Essentially take risks, expand your world model, but above all, don’t die. There’s a tension there - like “what happens if I poke the bear” vs “this might get me killed.”
Post reply on HN