Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Figure 01 robot demos its OpenAI integration

twitter.com

Post reply on HN
Print view View original

51–60 of 116 posts

Re: Figure 01 robot demos its OpenAI integration

#51

Post by bigyikes »

andoando wrote: post #39
DalasNoin wrote: post #6

Earlier quoted context omitted.

Similar at 1:47 I... I think It sounds so human, a person would also stutter at an introspective question like this. I wonder if their text to speech was trained on human data and produces these artifacts of human speech, or if it is intentional.

I would have added umms and hmms artificially just to make the latency less apparent, so Id say good chance thats what they did lol

Now that you mention it, I, uh, also add umms when my speech pathways experience high latency.

Re: Figure 01 robot demos its OpenAI integration

#53

Post by jxy »

margorczynski wrote: post #43
tomcam wrote: post #28

Earlier quoted context omitted.

I’ll be thrilled when Siri can spell my wife’s name correctly after 13 years of continuous usage and explicitly training it in her name. Admittedly her name is wildly complicated and totally unknown to the software folks at Apple: Ada Also my radio-trained voice is so generic a caller every week-ish assumes I am a bot, so I’m pretty sure the problem isn’t me enunciation or accent.

LLMs can't really spell as the smallest "block" of information they operate on are tokenized whole words. Same issue with e.g. arithmetic.

Can we stop this kind of misinformation? Training a model to map a token to individual letters are no harder than training a model to be fluent at English. Arithmetics with small number of digits are achievable as well. You can just try a small 7B model yourself. If you don't know where to start, try the mistral instruct v0.2, and this is how it goes,

> [INST] Spell out the following word letter by letter: margorczynski [/INST] m - a - r - g - o - r - c - z - y - n - s - k - i

> So, the word "margorczynski" spelled out letter by letter is: m-a-r-g-o-r-c-z-y-n-s-k-i.

The text between `[INST]` and `[/INST]` is the input. The text after `[/INST]` is the output.

Re: Figure 01 robot demos its OpenAI integration

#55

Post by fragmede »

modeless wrote: post #46

Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…

The Ok-robot demo shows that the technology for it to be fairly general is there, though no idea if figure one is using their technology or not. Simply being able to command a robot instead of moving a turtle with gcode is nothing short of astounding to those who aren’t deeply involved and tracking the sota progress in this area.

https://ok-robot.github.io/

Re: Figure 01 robot demos its OpenAI integration

#56

Post by Animats »

modeless wrote: post #46

Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…

> Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking.

Yes. Compare "Put That There" (1979).[1]

> The way that guy is standing perfectly still makes me think that it would fail if everything wasn't set up just so. I'd like to see demos with more variation.

Yes. Unstructured manipulation is hard. Structured robotic manipulation is pretty standard. Picking isolated objects is a solved problem. Here's a robot recycling sorting system, "Max-AI".[2] That's been in use for years. San Francisco recycling uses those robots. So do many other cities.

(That's from "Bulk Handling Systems", a company which does exactly what their name says. Recycling and trash come in bulk, and their machines handle it. Shakers, magnets, screens, and vision-based air sorters do 95% of the sorting. The robots only handle the hard cases. This is the no-bullshit end of AI.)

[1] https://www.youtube.com/watch?v=RyBEUyEtxQo

[2] https://max-ai.com/

Re: Figure 01 robot demos its OpenAI integration

#58

Post by userabchn »

One of my colleagues predicted, when ChatGPT was first released, that AI would reduce the value of knowledge work relative to manual labour, but I disagreed as I think the main thing holding back robots from replacing many manual labour jobs at the moment is the difficulty communicating with them. I argued that ChatGPT indicated that we were not too far away from being able to tell a robot to pick the apples from a particular row and put them in the green barn as the usual red barn was being painted today. This video suggests that I was right, and I in fact suspect that such manual labour is a more realistic type of work for AI (with robots) to substantially replace first than most knowledge work, where I think it will remain as an assistant for some time.

Re: Figure 01 robot demos its OpenAI integration

#59

Post by darth_avocado »

daveguy wrote: post #14

The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".

I am 100% convinced that the demo is partly fabricated, or at least very far from the robot’s capabilities. The inference part was probably true, but the dexterity would put them waaaaay ahead of what robots are capable of, if not programmed finely for a specific task. Industrial robots obviously have that kind of dexterity and precision because they’re specifically programmed for a task. General purpose robots however are nowhere as close to this level of accuracy or fluidity in their movements.

Re: Figure 01 robot demos its OpenAI integration

#60

Post by margorczynski »

jxy wrote: post #53
margorczynski wrote: post #43

Earlier quoted context omitted.

LLMs can't really spell as the smallest "block" of information they operate on are tokenized whole words. Same issue with e.g. arithmetic.

Can we stop this kind of misinformation? Training a model to map a token to individual letters are no harder than training a model to be fluent at English. Arithmetics with small number of digits are achievable as well. You can just try a small 7B model yourself. If you don't know where to start, try the mistral instruct v0.2, and this is how it goes, > [INST] Spell out the following word letter by letter: margorczyn…

Is Karpathy lying saying that word tokenization brings such problems that can be seen in many LLMs?

https://twitter.com/karpathy/status/1657949234535211009

I'm not arguing that you can't use single chars just that many of the issues parent discussed are caused by this.

Post reply on HN