Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Figure 01 robot demos its OpenAI integration

twitter.com

Post reply on HN
Print view View original

61–70 of 116 posts

Re: Figure 01 robot demos its OpenAI integration

#61

Post by ilaksh »

DalasNoin wrote: post #6
superzamp wrote: post #3

Interesting stutter at 0:53

Similar at 1:47 I... I think It sounds so human, a person would also stutter at an introspective question like this. I wonder if their text to speech was trained on human data and produces these artifacts of human speech, or if it is intentional.

I believe it's Eleven Labs API with the Stability setting turned down a little bit. It is definitely trained on human speech and when you use a somewhat lower setting than default, it will insert those types of natural imperfections or pauses and is very realistic.

Re: Figure 01 robot demos its OpenAI integration

#62

Post by iamflimflam1 »

userabchn wrote: post #58

One of my colleagues predicted, when ChatGPT was first released, that AI would reduce the value of knowledge work relative to manual labour, but I disagreed as I think the main thing holding back robots from replacing many manual labour jobs at the moment is the difficulty communicating with them. I argued that ChatGPT indicated that we were not too far away from being able to tell a robot to pick the apples from a p…

I think we are going to find that the Moravec paradox is wrong. Interesting times…

Re: Figure 01 robot demos its OpenAI integration

#63

Post by mjfisher »

lelag wrote: post #27

It's a really cool demo and I'm impressed by the dexterity of the robot, however I'm a bit underwhelm by what's shown here in the sense that the speech and reasoning capabilities is just obvious to anyone who's been paying attention and has experience with GPT4. The function calling was great, but it had a very simple "world" to interact with. It's really interesting to see it integrated with a robot that can interac…

I take your point about familiarity with GPT4 making this less immediately impressive - but just as an end to end demo it's absolutely mind-blowing how far we've come.

Can you imagine seeing this ten years ago? Moving so far in such a short time frame would have been unbelievable.

This gave me the "I'm actually living in the future" vibes that I always imagined I'd get from flying cars.

Re: Figure 01 robot demos its OpenAI integration

#64

Post by educaysean »

modeless wrote: post #13
eigenvalue wrote: post #11

This shows the real utility of the Groq low latency inference. That delay in responding makes it much less impressive (it’s still really impressive obviously!)

That delay will be eliminated very soon. IMO low latency natural voice conversations are going to be bigger than ChatGPT. It's going to blow people's minds when they can converse with these AIs casually just like with their real life friends. It won't be anything like Siri or Alexa anymore. Here's a demo from a startup in this space. Still very early. https://deck.sindarin.tech/

I must say the demo did nothing to improve my opinion of the current state of voice-based AI conversations.

Re: Figure 01 robot demos its OpenAI integration

#65

Post by whatever1 »

darth_avocado wrote: post #59
daveguy wrote: post #14

The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".

I am 100% convinced that the demo is partly fabricated, or at least very far from the robot’s capabilities. The inference part was probably true, but the dexterity would put them waaaaay ahead of what robots are capable of, if not programmed finely for a specific task. Industrial robots obviously have that kind of dexterity and precision because they’re specifically programmed for a task. General purpose robots howev…

Probably some sort of imitation learning

Re: Figure 01 robot demos its OpenAI integration

#67

Post by StarCyan »

huytersd wrote: post #49
famouswaffles wrote: post #34

Earlier quoted context omitted.

The robot movements are separate end-to-end neural networks. They're triggered by the LLM but aren't hardcoded.

Do you know this for a fact or are you speculating?

One of the Figure engineers described the system at a high level: https://twitter.com/coreylynch/status/1767927506332721154

Re: Figure 01 robot demos its OpenAI integration

#68

Post by StarCyan »

daveguy wrote: post #14

The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".

Yeah the motion looks way more fluid then similar systems I've seen before.

Re: Figure 01 robot demos its OpenAI integration

#69

Post by ilaksh »

gotrythis wrote: post #29
niek_pas wrote: post #21

Earlier quoted context omitted.

Funny, I have the exact opposite response. It does, indeed, make it seem more human-like to me.

I agree. That's the sad part. I want super-intelligence to sound intelligent and not artificially brought down to our level. :-)

If they are using Eleven Labs, this is just the Stability setting. Turning it down will make it more realistic and closer to the training data. That is what causes the pauses and imperfections.

You can sign up and use their Voice Lab for free or maybe a few bucks and experiment with the slider for Stability and the other setting.

In my opinion, turning Stability down just a little bit to demo extremely realistic speech is a no-brainer. They could have turned it up and made it ultra-smooth, but that makes no sense. Why make your robot demo less realistic deliberately?

Re: Figure 01 robot demos its OpenAI integration

#70

Post by fragmede »

andoando wrote: post #37
tomp wrote: post #23

Earlier quoted context omitted.

IME I can get about 0.2s to get the first chunk from Mistral (i.e. Mistral API, using Mixtral model (`mistral-small`), not Mixtral on Groq) (and note the Mistral sends larger chunks, unlike ChatGPT which sends individual tokens) and another 0.6s or so to get first voice chunks from PlayHT measuring STT latency is harder, I need to implement a local VAD model first to properly measure it, but I think it's on the order…

just add a hmmm before every response

Which we humans do all the time. Okay, like, so, hear me out, alright? See, what’s really going on, yeah, is…
Post reply on HN