Earlier quoted context omitted.
That's just about any video of any VLA ever. Including Gemini Robotics 2.
Should be trivial to link to one then...
Off the top of my head:
Gemini Robotics 2 brings whole body intelligence to robots
571–580 of 583 posts
Re: Gemini Robotics 2 brings whole body intelligence to robots
#572Earlier quoted context omitted.
Should be trivial to link to one then...
Off the top of my head: https://www.pi.website/blog/pi07
As I suspected you are fooled by this video and imagine it to be capable of much more than what is shown. This video is pretty non-marketing and is quite straight to the point. But that does not prevent you from being awed!
So What is LLM is used here for? It is used for mere translation between different robots. So it is mostly symbolic translation.
What I am talking about is to translation LLM inference directly to movements. For example, if you ask an LLM, how do I open the microwave door? It will list the steps. I am talking about a system that can go from "put the thing in the microwave", to action steps, without having to never once demonstrate it physically, and do it just from LLM inference.
In short, the way LLMs used here is not (categorically) the way I was asking about.
Re: Gemini Robotics 2 brings whole body intelligence to robots
#573Earlier quoted context omitted.
Off the top of my head: https://www.pi.website/blog/pi07
As I suspected you are fooled by this video and imagine it to be capable of much more than what is shown. This video is pretty non-marketing and is quite straight to the point. But that does not prevent you from being awed! So What is LLM is used here for? It is used for mere translation between different robots. So it is mostly symbolic translation. What I am talking about is to translation LLM inference directly to…
Read. The. Papers.
https://arxiv.org/pdf/2505.23705
https://www.pi.website/download/pistar06.pdf
https://www.pi.website/download/pi07.pdf
The thing literally has a diffusion "action expert" sit in the same attention system as a pre-trained VLM. And the VLM itself is ALSO trained to generate raw actions as a part of the training recipe (the first paper) - it just doesn't do it at inference time. What the "action expert" does is parallelize the action generation process - based on VLM's internal states.
It's exactly the thing you claimed to be impossible. Described in detail in a paper from 2025. What's your excuse?
Re: Gemini Robotics 2 brings whole body intelligence to robots
#574Earlier quoted context omitted.
As I suspected you are fooled by this video and imagine it to be capable of much more than what is shown. This video is pretty non-marketing and is quite straight to the point. But that does not prevent you from being awed! So What is LLM is used here for? It is used for mere translation between different robots. So it is mostly symbolic translation. What I am talking about is to translation LLM inference directly to…
Read. The. Papers. https://arxiv.org/pdf/2505.23705 https://www.pi.website/download/pistar06.pdf https://www.pi.website/download/pi07.pdf The thing literally has a diffusion "action expert" sit in the same attention system as a pre-trained VLM. And the VLM itself is ALSO trained to generate raw actions as a part of the training recipe (the first paper) - it just doesn't do it at inference time. What the "action exper…
I have not overlooked anything. I had imagined that this "mapping", to have any power, would also need to be handled by an LLM (action expert). But here is the problem with that. That would not be as "intelligent" as an LLM....And you can't make it as smart as the LLM because there is not a similarly huge training data on which LLMs are trained on..
Re: Gemini Robotics 2 brings whole body intelligence to robots
#575Earlier quoted context omitted.
Read. The. Papers. https://arxiv.org/pdf/2505.23705 https://www.pi.website/download/pistar06.pdf https://www.pi.website/download/pi07.pdf The thing literally has a diffusion "action expert" sit in the same attention system as a pre-trained VLM. And the VLM itself is ALSO trained to generate raw actions as a part of the training recipe (the first paper) - it just doesn't do it at inference time. What the "action exper…
I have not overlooked anything. I had imagined that this "mapping", to have any power, would also need to be handled by an LLM (action expert). But here is the problem with that. That would not be as "intelligent" as an LLM....And you can't make it as smart as the LLM because there is not a similarly huge training data on which LLMs are trained on..
The backbone of the VLA there is literally a pre-trained Gemma model. And a small one at that.
You already downgraded your claims from "LLMs are irrelevant to robotics" to a measly "you can't train a useful robotics LLM because there's not enough data". And you say that while looking at an LLM that was pre-trained on all of internet scraped and only then reused for robotics.
Both the pool of robotics-relevant data and the performance of foundation model LLMs grow over time. All the companies that are serious about robotics are serious about scaling up data collection.
I'm not going to claim that this "LLM core" approach is the best approach to AI robotics possible - but if you're betting on it failing outright, you're going to be fighting uphill.
Re: Gemini Robotics 2 brings whole body intelligence to robots
#576Earlier quoted context omitted.
I have not overlooked anything. I had imagined that this "mapping", to have any power, would also need to be handled by an LLM (action expert). But here is the problem with that. That would not be as "intelligent" as an LLM....And you can't make it as smart as the LLM because there is not a similarly huge training data on which LLMs are trained on..
The backbone of the VLA there is literally a pre-trained Gemma model. And a small one at that. You already downgraded your claims from "LLMs are irrelevant to robotics" to a measly "you can't train a useful robotics LLM because there's not enough data". And you say that while looking at an LLM that was pre-trained on all of internet scraped and only then reused for robotics. Both the pool of robotics-relevant data an…
>to a measly "you can't train a useful robotics LLM because there's not enough data"..
This was the claim from the very beginning. You should have asked why I think what I think, instead of leading with "the entire premise is wrong!"...
Re: Gemini Robotics 2 brings whole body intelligence to robots
#577Earlier quoted context omitted.
What innovations specifically?
Can't really name them off the top of my head, sorry! But if you look up on Google or X, there are multiple companies doing things like pressure based actuators (Clone robotics) or mesh-based knit ones, new forms of EFAM, some new soft actuators and more.
Yeah, these types of actuators are pretty interesting, but I've not seen anything super promising yet.
Re: Gemini Robotics 2 brings whole body intelligence to robots
#578Earlier quoted context omitted.
The backbone of the VLA there is literally a pre-trained Gemma model. And a small one at that. You already downgraded your claims from "LLMs are irrelevant to robotics" to a measly "you can't train a useful robotics LLM because there's not enough data". And you say that while looking at an LLM that was pre-trained on all of internet scraped and only then reused for robotics. Both the pool of robotics-relevant data an…
>to a measly "you can't train a useful robotics LLM because there's not enough data".. This was the claim from the very beginning. You should have asked why I think what I think, instead of leading with "the entire premise is wrong!"...
No, you openly, plainly went and downgraded your claim to a somewhat defensible one. It's not subtle.
Your entire premise was wrong at every point, and now you're trying to wriggle your way out of admitting it.
Re: Gemini Robotics 2 brings whole body intelligence to robots
#579Earlier quoted context omitted.
>to a measly "you can't train a useful robotics LLM because there's not enough data".. This was the claim from the very beginning. You should have asked why I think what I think, instead of leading with "the entire premise is wrong!"...
No, you openly, plainly went and downgraded your claim to a somewhat defensible one. It's not subtle. Your entire premise was wrong at every point, and now you're trying to wriggle your way out of admitting it.
> No, you openly, plainly went and downgraded your claim..
Prove it!
Re: Gemini Robotics 2 brings whole body intelligence to robots
#580Earlier quoted context omitted.
I'm repeating "what you claim to be impossible was done 3 years ago and was already replaced with better versions of the same idea and you are hilariously out of touch".
Show me one video where a robot follows textual prompts and come up with its own movements to solve the prompt.
https://huggingface.co/blog/lerobot-release-v060#molmoact2
Read the command line prompt: --task="pick up the red cube"
There is a gif directly below it.
This is a completely open source model and arm you can replicate yourself.