Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

youtube.com

Post reply on HN
Print view View original

121–130 of 163 posts

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#121

Post by jandrewrogers »

calf wrote: post #114
jandrewrogers wrote: post #99

Earlier quoted context omitted.

I'm not aware of any convenient literature but it is relatively obvious once someone explains it to you (as it was explained to me). At its root it is a cutting problem, like graph cutting but much more general because it includes things like non-trivial geometric types and relationships. Solving the cutting problem is necessary to efficiently shard/parallelize operations over the data models. For classic scalar data…

But then that sounds more like that person explained it wrong. They didn't explain why it is necessary to reduce to GRAPHCUT, it seems to me to beg the question. We should not assume this is true based on some vague anthropomorphic appeal to spatial locality, surely?

It isn’t a graph cutting problem, graph cutting is just a simpler, special case of this more general cutting problem (h/t IBM Research). If you can solve the general problem you effectively get efficient graph cutting for free. This is obviously attractive to the extent you can do both complex spatial and graph computation at scale on the same data structure instead of specializing for one or the other.

The challenge with cutting e.g. rectangles into uniform subsets is that logical shard assignment must be identical regardless of insertion order and in the absence of an ordering function, with O(1) space complexity and without loss of selectivity. Arbitrary sets of rectangles overlap, sometimes heavily, which is the source of most difficulty.

Of course, with practical implementations write scalability matters and incremental construction is desirable.

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#123

Post by weinzierl »

Workaccount2 wrote: post #70
weinzierl wrote: post #43

Earlier quoted context omitted.

Condensing it for HN was harder than I thought because most of it makes only sense when you also see the images, so here is more like a summary of parts of the dialogue. Prompted by this comment https://news.ycombinator.com/item?id=44366753 I tried to geolocate the camera. I uploaded a screenshot from https://walzr.com/weather-watching to ChatGPT and it said a lot of things but concluded with “New York City street co…

I'd take text-to-image capabilities with a grain of salt, because they are dramatically lower than their text to text abilities. I don't know the exact mechanics with current multimodal models, but it is pretty clear that there is a disconnect between what the text modal wants, and what the text model outputs. It's almost feels like asking someone with a blindfold to draw a cat, you kinda get a mess. If you ask chatg…

This is a good point. The 2D birds eye view image adds another separate complication. There are certainly better and more direct ways to show that current models are bad with spatial reasoning. This was just a byproduct of my geolocation experiments. Maybe I will give it a shot another day.

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#124

Post by mumbisChungo »

dopadelic wrote: post #107
jandrewrogers wrote: post #8

I appreciate the video and generally agree with Fei-Fei but I think it almost understates how different the problem of reasoning about the physical world actually is. Most dynamics of the physical world are sparse, non-linear systems at every level of resolution. Most ways of constructing accurate models mathematically don’t actually work. LLMs, for better or worse, are pretty classic (in an algorithmic information t…

You're pointing out a real class of hard problems — modeling sparse, nonlinear, spatiotemporal systems — but there’s a fundamental mischaracterization in lumping all transformer-based models under “LLMs” and using that to dismiss the possibility of spatial reasoning. Yes, classic LLMs (like GPT) operate as sequence predictors with no inductive bias for space, causality, or continuity. They're optimized for language f…

Claude, is that you?

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#125

Post by moktonar »

Workaccount2 wrote: post #66
moktonar wrote: post #58

Intelligence is not only embodied (it needs a body), it is also embedded in the environment (it needs the environment). If you want an intelligence in your computer, you need an environment in your computer first, as the substrate from which the intelligence will evolve. The more accurate the environment the better the intelligence that will be obtained. The universe is able to create intelligence and we are proof. T…

The silicon exists in the same environment that our brains do.

Yeah, and we represent the evolutionary force, but that means that the ability to craft silicon life depends on our ability to find an efficient algorithm to do so...

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#126

Post by voxleone »

jandrewrogers wrote: post #8

I appreciate the video and generally agree with Fei-Fei but I think it almost understates how different the problem of reasoning about the physical world actually is. Most dynamics of the physical world are sparse, non-linear systems at every level of resolution. Most ways of constructing accurate models mathematically don’t actually work. LLMs, for better or worse, are pretty classic (in an algorithmic information t…

I'm trying to approach spatial reasoning by introducing quaternions to navigate graphs. It is a change in the unit of traversal — from positional increment to rotational progression. This reframing has cascading effects. It alters how we model motion, how we think about proximity, and ultimately how systems relate to space itself.

The traditional metaphor of movement — stepping from point A to point B — is spatially intuitive but semantically impoverished. It ignores the continuity of direction, the embodiment of motion, and the nontriviality of turning. Quaternion-based traversal reintroduces these elements. It is not just more precise; it is more faithful to the mechanisms by which physical and virtual entities evolve through space. In other words objects 'become' the model.

https://github.com/VoxleOne/SpinStep/blob/main/docs/index.md

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#127

Post by SlowTao »

whiplash451 wrote: post #98
skwb wrote: post #3

It's hard to describe, but it's felt like LLMs have completely sucked the entire energy out of computer vision. Like... I know CVPR still happens and there's great research that comes out of it, but almost every single job posting in ML is about LLMs to do this and that to the detriment of computer vision.

It felt the same back in 2012-2015 when deep learning was flooding over computer vision. Yet 10 years later there is a net benefit for computer vision: a lot of tasks are now solved much better/more efficiently with deep learning including those that seemed "unfit" to deep learning like tracking. I'm hopeful that VLMs will "fan out" into a lot of positive outcomes for computer vision.

That is fair. I think it is a case of just seeing a lot if great talent rush to the "in" thing. Other systems are still being developed and that isnt lost but there is just a feeling if being left out of it all while still doing great stuff.

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#128

Post by tehjoker »

jandrewrogers wrote: post #99
teemur wrote: post #42

Earlier quoted context omitted.

> We know that universal solutions can’t exist and that all practical solutions require exotic high-dimensionality computational constructs that human brains will struggle to reason about. This has been the status quo since the 1980s. This particular set of problems is hard for a reason. This made me a bit curious. Would you have any pointers to books/articles/search terms if one wanted to have a bit deeper look on t…

I'm not aware of any convenient literature but it is relatively obvious once someone explains it to you (as it was explained to me). At its root it is a cutting problem, like graph cutting but much more general because it includes things like non-trivial geometric types and relationships. Solving the cutting problem is necessary to efficiently shard/parallelize operations over the data models. For classic scalar data…

Did that research program have a public code name?

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#129

Post by mindcrime »

tehjoker wrote: post #128
jandrewrogers wrote: post #99

Earlier quoted context omitted.

I'm not aware of any convenient literature but it is relatively obvious once someone explains it to you (as it was explained to me). At its root it is a cutting problem, like graph cutting but much more general because it includes things like non-trivial geometric types and relationships. Solving the cutting problem is necessary to efficiently shard/parallelize operations over the data models. For classic scalar data…

Did that research program have a public code name?

Looking through some old DARPA budget docs[1], it seems like there's a chance that what's being discussed here falls under DARPA's "PE 0602702E TACTICAL TECHNOLOGY" initiative, project TT-06.

Some other possibilities might include:

  - "PE 0602304E COGNITIVE COMPUTING SYSTEMS", project COG-02.
  - "PE 0602716E ELECTRONICS TECHNOLOGY", project ELT-01
  - "PE 0603760E COMMAND, CONTROL AND COMMUNICATIONS SYSTEMS", project CCC-02
  - "PE 0603766E NETWORK-CENTRIC WARFARE TECHNOLOGY", project NET-01
  - "PE 0603767E SENSOR TECHNOLOGY", project SEN-02
Or maybe it's nothing to do with this at all. But in either case, this looks like some interesting stuff to explore in its own right. :-)

[1]: https://web.archive.org/web/20181001000000/https://www.darpa...

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#130

Post by m463 »

IdealeZahlen wrote: post #20

I've always wondered how spatial reasoning appears to be operating quite differently from other cognitive abilities, with significant individual variations. Some people effortlessly parallel park while others struggle with these tasks despite excelling at other forms of pattern recognition. What was particularly intriguing for me is that some people with aphantasia have no difficulty with spatial reasoning tasks, so…

I have had this idea about parking a car...

Most people have proprioception - you know where the parts of your body are without looking. Close your eyes and you intuitively know where your hands and fingers are.

When parking a car, it helps to sort of sit in the drivers seat and look around the car. Turn your neck and look past the back seat where your rear tire would be. sense the edges of the car.

I think if you sort of develop this a bit you might "feel" where your car is intuitively when pulling into a parking space or parallel parking. (car-prioception?)

(but use your mirrors and backup camera anyway)

Post reply on HN