Live data from Hacker News

Figure 01 robot demos its OpenAI integration

twitter.com

31–40 of 116 posts

Re: Figure 01 robot demos its OpenAI integration

#32
post #14

The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".

How though. It’s probably just predefined actions that are triggered by the LLM output. At the same time it would be impressive if the LLM determined the right function to call in real time, was able to deal with the ambiguity in placement of the garbage and bonus points if it could do that in a scenario that wasn’t hardcoded to exactly standing behind that table in that spot.

Re: Figure 01 robot demos its OpenAI integration

#33
post #26

I found this discussion interesting. Talking about this as a new form of labor. https://twitter.com/CernBasher/status/1767939757105991791

Reads like an ad written by GPT, honestly.

But this guy is a professional advice giver, so to be expected?

Wouldn’t surprise me if he outsourced his tweeting.

Re: Figure 01 robot demos its OpenAI integration

#34
post #14

The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".

How though. It’s probably just predefined actions that are triggered by the LLM output. At the same time it would be impressive if the LLM determined the right function to call in real time, was able to deal with the ambiguity in placement of the garbage and bonus points if it could do that in a scenario that wasn’t hardcoded to exactly standing behind that table in that spot.

The robot movements are separate end-to-end neural networks. They're triggered by the LLM but aren't hardcoded.

Re: Figure 01 robot demos its OpenAI integration

#35
post #23

This shows the real utility of the Groq low latency inference. That delay in responding makes it much less impressive (it’s still really impressive obviously!)

IME I can get about 0.2s to get the first chunk from Mistral (i.e. Mistral API, using Mixtral model (`mistral-small`), not Mixtral on Groq) (and note the Mistral sends larger chunks, unlike ChatGPT which sends individual tokens) and another 0.6s or so to get first voice chunks from PlayHT measuring STT latency is harder, I need to implement a local VAD model first to properly measure it, but I think it's on the order…

Unless the only thing you want to do with the robot is talk, you need to do a lot more reasoning and execution planning first (= multiple LLM round trips; tool calling) before you even know whether talking is the correct action to take. So the naive time-to-first-chunk estimate will be way off.

Re: Figure 01 robot demos its OpenAI integration

#36
post #14

The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".

Is it just me or are people easily impressed by these robot demo that moves nothing like humans and doing the simplistic tasks like passing a large object. The freaking Apple is within his arms reach and he asked to pass the Apple. I almost laughed out loud.

Re: Figure 01 robot demos its OpenAI integration

#37
post #23

This shows the real utility of the Groq low latency inference. That delay in responding makes it much less impressive (it’s still really impressive obviously!)

IME I can get about 0.2s to get the first chunk from Mistral (i.e. Mistral API, using Mixtral model (`mistral-small`), not Mixtral on Groq) (and note the Mistral sends larger chunks, unlike ChatGPT which sends individual tokens) and another 0.6s or so to get first voice chunks from PlayHT measuring STT latency is harder, I need to implement a local VAD model first to properly measure it, but I think it's on the order…

just add a hmmm before every response

Re: Figure 01 robot demos its OpenAI integration

#39

Interesting stutter at 0:53

Similar at 1:47 I... I think It sounds so human, a person would also stutter at an introspective question like this. I wonder if their text to speech was trained on human data and produces these artifacts of human speech, or if it is intentional.

I would have added umms and hmms artificially just to make the latency less apparent, so Id say good chance thats what they did lol
Post reply on HN