Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

youtube.com

Post reply on HN
Print view View original

71–80 of 163 posts

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#71

Post by m-s-y »

loa_in_ wrote: post #62
HappMacDonald wrote: post #55

Earlier quoted context omitted.

Yeah I'm envisioning putting a thousand simplistic robotic "infants" into a vast "playpen" to gather sensor data about their environment, for some (probably smaller) number of deep learning models to ingest the input and guess at output strategies (move this servo, rotate this camshaft this far in that direction, etc) and make predictions about resulting changes to input. In principle a thousand different deep learni…

But the playpen will contain objects that are inherently breakable. You cannot rough handle the glass vessel and have it too.

The world Is breakable. Any model based on it will need to know this anyway. Am I missing your argument?

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#72

Post by m3kw9 »

skwb wrote: post #3

It's hard to describe, but it's felt like LLMs have completely sucked the entire energy out of computer vision. Like... I know CVPR still happens and there's great research that comes out of it, but almost every single job posting in ML is about LLMs to do this and that to the detriment of computer vision.

There is nothing to productize vs LLMs right now. I would say robots could fix that but they have hard problems to solve in the physical sense that will bottle neck things

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#73

Post by csullivan107 »

HappMacDonald wrote: post #55
psb217 wrote: post #41

Earlier quoted context omitted.

I think there's an implicit assumption here that interaction with the world is critical for effective learning. In that case, you're bottlenecked by the speed of the world... when learning with a single agent. One neat thing about artificial computational agents, in contrast to natural biological agents, is that they can share the same brain and share lived experience, so the "speed of reality" bottleneck is much les…

Yeah I'm envisioning putting a thousand simplistic robotic "infants" into a vast "playpen" to gather sensor data about their environment, for some (probably smaller) number of deep learning models to ingest the input and guess at output strategies (move this servo, rotate this camshaft this far in that direction, etc) and make predictions about resulting changes to input. In principle a thousand different deep learni…

Never thought I’d get to do this but this was my masters research! Simulations are inherently limited and I just got tired of robotic research being done only in simulations. So I built a novel soft robot (notoriously difficult to control) and got it to learn by playing!!

Here is an informal talk I gave on my work. Let me know if you want the thesis

https://www.youtube.com/live/ZXlQ3ppHi-E?si=MKcRqoxmEra7Zrt5

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#74

Post by TheOtherHobbes »

lsy wrote: post #22
jandrewrogers wrote: post #8

I appreciate the video and generally agree with Fei-Fei but I think it almost understates how different the problem of reasoning about the physical world actually is. Most dynamics of the physical world are sparse, non-linear systems at every level of resolution. Most ways of constructing accurate models mathematically don’t actually work. LLMs, for better or worse, are pretty classic (in an algorithmic information t…

To make this more concrete: ImageNet enabled computer "vision" by providing images + labels, enabling the computer to take an image and spit out a label. LLM training sets enable text completion by providing text + completions, enabling the computer to take a piece of text and spit out its completion. Learning how the physical world works (not just kind of works a la videogames, actually works ) is not only about a j…

Considering how bad LLMs are at understanding anything, and how they still manage to be useful, you simply don't need this level of complexity.

You need something that mostly works most of the time, and has guardrails so when it makes mistakes nothing bad happens.

Our brains acquire quite good heuristics for dealing with physical space without needing to experience all of physical reality.

A cat-level or child-level understanding of physical space is more immediately useful than a philosopher-level of understanding.

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#75

Post by fu-hn »

calf wrote: post #47
machinelearning wrote: post #13

Earlier quoted context omitted.

"Most ways of constructing accurate models mathematically don’t actually work" > This is true for almost anything at the limit, we are already able to model spatiotemporal dynamics to some useful degree (see: progress in VLAs, video diffusion, 4D Gaussians) "We’ve known for well over a decade that you cannot cram real-world spatial dynamics into those models. It is a clear impedance mismatch" > What's the source that…

Above commenter also asserts "highly developed research but no public literature" shrug ...

But the best minds in the world said so!

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#76

Post by gk1 »

District5524 wrote: post #38

An immaterial side note: funny how obsessed she seems to be with her age. She said once that people in the audience could be half or even third of her age. Given that she's 49, is it really typical that 16-year olds attend these fireside YC chats?

I only noticed a few times when she brought up age, and all of them were natural and appropriate in the context. Eg she was asked how old she was when she opened the dry-cleaning business.

Also, there’s this weird thing in culture (is it US only?) that whenever an interviewer brings up (even implicitly) the guest’s age, the guest has to make some quip about it as if they’re offended or sensitive to it. So I wouldn’t interpret even a slightly defensive comment about age as an “obsession.”

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#77

Post by dcreater »

moktonar wrote: post #58

Intelligence is not only embodied (it needs a body), it is also embedded in the environment (it needs the environment). If you want an intelligence in your computer, you need an environment in your computer first, as the substrate from which the intelligence will evolve. The more accurate the environment the better the intelligence that will be obtained. The universe is able to create intelligence and we are proof. T…

So your basically saying we need Dolores and Westworld

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#78

Post by owenpalmer »

While spatial intelligence is certainly a major limitation of current AI systems, I have been able to get LLMs to do quite impressive things.

Here's an on the fly video I made (no retakes) of Claude generating a Godot scene file.

https://youtu.be/2gARJpDG7Jo?si=W4rlISO-J4EPJYyG

Re: Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

#80

Post by [deleted] »

jandrewrogers wrote: post #8

I appreciate the video and generally agree with Fei-Fei but I think it almost understates how different the problem of reasoning about the physical world actually is. Most dynamics of the physical world are sparse, non-linear systems at every level of resolution. Most ways of constructing accurate models mathematically don’t actually work. LLMs, for better or worse, are pretty classic (in an algorithmic information t…

[deleted]
Post reply on HN