Helix: A vision-language-action model for generalist humanoid control
1–10 of 178 posts
Re: Helix: A vision-language-action model for generalist humanoid control
#2Wow! This is something new.
Re: Helix: A vision-language-action model for generalist humanoid control
#3YouTube link for the video (for whatever reason the video hosted on their site kept buffering for me): https://www.youtube.com/watch?v=Z3yQHYNXPws
Re: Helix: A vision-language-action model for generalist humanoid control
#4When doing robot control, how do you model in the control of the robot? Do you have tool_use / function calling at the top level model which then gets turned into motion control parameters via inverse kinematic controllers?
What is the interface from the top level to the motors?
I feel it can not just be a neural network all the way down, right?
Re: Helix: A vision-language-action model for generalist humanoid control
#5"The first time you've seen these objects" is a weird thing to say. One presumes that this is already in their training set, and that these models aren't storing a huge amount of data in their context, so what does that even mean?
Re: Helix: A vision-language-action model for generalist humanoid control
#6There’s nothing I want more than a robot that does house chores. That’s the real 10x multiplier for humans to do what they do best.
Re: Helix: A vision-language-action model for generalist humanoid control
#7Very impressive
Why make such sinister-looking robots though...?
Re: Helix: A vision-language-action model for generalist humanoid control
#8[deleted]
Re: Helix: A vision-language-action model for generalist humanoid control
#9There’s nothing I want more than a robot that does house chores. That’s the real 10x multiplier for humans to do what they do best.
Hopefully in the next decade we’ll get there.
Vision+language multimodal models seem to solve some of the hard problems.
Re: Helix: A vision-language-action model for generalist humanoid control
#10When doing robot control, how do you model in the control of the robot? Do you have tool_use / function calling at the top level model which then gets turned into motion control parameters via inverse kinematic controllers? What is the interface from the top level to the motors? I feel it can not just be a neural network all the way down, right?
Have a look at the post - it explains how it works. There are two models: a 7-9Hz 7B vision-language model, and a 200Hz 80M visuomotor model. The former produces a latent vector, which is then interpreted by the latter to drive the motors.