Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helix: A vision-language-action model for generalist humanoid control

figure.ai

Post reply on HN
Print view View original

161–170 of 178 posts

Re: Helix: A vision-language-action model for generalist humanoid control

#161

Post by imtringued »

UltraSane wrote: post #148
causal wrote: post #67

I'm always wondering at the safety measures on these things. How much force is in those motors? This is basically safety-critical stuff but with LLMs. Hallucinating wrong answers in text is bad, hallucinating that your chest is a drawer to pull open is very bad.

You can have dedicated controllers for the motors that limit their max torque.

That's not enough. When a robot link is in motion and hits an object, the change in momentum creates an impulse over the duration of deceleration. The faster the robot moves, the faster it has to decelerate, the higher the instantaneous braking force at the impact point.

Re: Helix: A vision-language-action model for generalist humanoid control

#162

Post by imtringued »

01100011 wrote: post #86
ziofill wrote: post #6

There’s nothing I want more than a robot that does house chores. That’s the real 10x multiplier for humans to do what they do best.

I'd pay $2k for something that folds my laundry reliably. It doesn't need arms or legs, just like my dishwasher doesn't need arms or legs. It just needs to let me dump in a load of clean laundry and output stacks of neatly folded or hung clothing.

Laundry folding machines already exist. You can find cheap ones on AliExpress.

https://www.aliexpress.com/w/wholesale-clothes-folding-machi...

Re: Helix: A vision-language-action model for generalist humanoid control

#163

Post by imtringued »

swalsh wrote: post #60

At this point, this is enough autonomy to have a set of these guys man a howitzer (read as old stockpiles of weapons we already have). Kind of a scary thought. On one hand, I think the idea of moving real people out of danger in war is a good idea, and as an American i'd want Americans to have an edge... and we can't guarantee our enemies won't take it if we skip it, on the other hand I have a visceral reaction to ma…

Panzerhaubitze 2000 already has an autoloader and the entire point of self propelled artillery is that it moves after shooting to avoid counter artillery fire.

Re: Helix: A vision-language-action model for generalist humanoid control

#164

Post by imtringued »

bhouston wrote: post #4

When doing robot control, how do you model in the control of the robot? Do you have tool_use / function calling at the top level model which then gets turned into motion control parameters via inverse kinematic controllers? What is the interface from the top level to the motors? I feel it can not just be a neural network all the way down, right?

You don't use function calling. You specifically train the neural network to directly encode the robot action as a token. There are many ways. You can output absolute positions, delta positions, relative trajectory. You can do this in joint space or end effector space.

200Hz is barely enough to control a motor, but it is good enough to send a reference signal to a motor controller. Usually what is done is that you have a neural network to learn complex high level behaviour and use that to produce a high level trajectory, then you have a whole body robot controller based on quadratic programming that does things like balancing, maintaining contacts when holding objects or pressing against things. This requires a model of the robot dynamics so that you know the relationship between torques and acceleration. Then after that you will need a motor controller that accepts reference acceleration/torque, velocity and position commands which then is turned into 10kHz to 100kHz pulse width modulated signals by the motor controller. The motor controller itself is driving MOSFETs so it can only turn them on or off, unless you are using expensive sinusoidal drivers.

Re: Helix: A vision-language-action model for generalist humanoid control

#165

Post by matteocontrini »

andiareso wrote: post #104

Seriously, what's with all of these perceived "high-end" tech companies not doing static content worth a damn. Stop hosting your videos as MP4s on your web-server. Either publish to a CDN or use a platform like YouTube. Your bandwidth cannot handle serving high resolution MP4s. /rant

What do you mean? Videos on that page are served by CloudFront. If you're seeing issues it may be that videos are not encoded for web playback (faststart, etc.) but I haven't checked.

Re: Helix: A vision-language-action model for generalist humanoid control

#166

Post by anentropic »

esafak wrote: post #42
anentropic wrote: post #7

Very impressive Why make such sinister-looking robots though...?

Black was not the best color choice.

Well, when you put it like that I feel a bit uncomfortable...

But it did seem like title of their mood board must have been "Black Mirror".

Very uncanny valley, the glossy facelessness. It somehow looks neither purely utilitarian/industrial nor 'friendly'. I could see it being based on the aesthetic of laptops and phones, i.e. consumer tech, but the effect is so different when transposed onto a very humanoid form.

Re: Helix: A vision-language-action model for generalist humanoid control

#167

Post by idiotsecant »

UltraSane wrote: post #149
causal wrote: post #80

Earlier quoted context omitted.

Are you saying it cannot move faster than they because of some kind of governor?

That is how I would design it. It is common in safety critical PLC systems to have 1 or more separate safety PLCs that try to prevent bad things from happening.

Although in a SIL safety system the dangerous events are identified and extremely thoroughly characterized as part of system design.

There cannot be a safety system of this type for a generalist platform like a humanoid robot. It's possibility space is just too high.

I think the safety governor in this case would have to be a neural network that is at least as complex as the robots network, if not more so.

Which begs the question: what system checks that one for safety?

Re: Helix: A vision-language-action model for generalist humanoid control

#168

Post by rtkwe »

kilroy123 wrote: post #152
UltraSane wrote: post #147

Earlier quoted context omitted.

That could still be decades away.

I don't know... I'm starting to seriously think that is only 5-10 years away.

Is that a real 5-10 years, a research 5-10 years[0] or 5-10 years of "FSD in the next 6 months"?

The demo space is so sterile and empty I think we're still a loong ways off from the Coffee test happening. One big thing I see is they don't have to rearrange other items they have nice open bins/drawers/shelves/etc to drop the items into. That kind of multistep planning has been a thorn in independent robotics for decades.

[0] https://xkcd.com/678/

Re: Helix: A vision-language-action model for generalist humanoid control

#169

Post by Symmetry »

selfmodruntime wrote: post #131
Symmetry wrote: post #95

Earlier quoted context omitted.

They can put away clutter but if they could chop a carrot or dust a vase they'd have shown videos demonstrating that sort of capability. EDIT: Let alone chop an onion. Let me tell you having a robot manipulate onions is the worst. Dealing with loose onion skins is very hard.

Sure. But if you showed this video to someone 5 or 10 years ago, they'd say it's fiction.

Telling a robot verbally "Put the cup on the counter" and having it figure out what the cup is, what the counter is in its field of view would have seemed like science fiction. The object manipulation itself is still well behind what we saw in the 2015 DARPA Grand Challenge though.

Re: Helix: A vision-language-action model for generalist humanoid control

#170

Post by rtkwe »

causal wrote: post #67

I'm always wondering at the safety measures on these things. How much force is in those motors? This is basically safety-critical stuff but with LLMs. Hallucinating wrong answers in text is bad, hallucinating that your chest is a drawer to pull open is very bad.

That's actually more of a solved problem. Robot arms that can track the force they're applying and where to avoid injuring humans have been kicking around for 10-15 years. It let them go out of the mega safety cells into the same space as people and even do things like letting the operator pose the robot to teach it positions instead of having to do it in a computer program or with a remote control.

The term I see a lot is co-robotics or corobots. At least that's what Kuka calls them.

Post reply on HN