It's a really cool demo and I'm impressed by the dexterity of the robot, however I'm a bit underwhelm by what's shown here in the sense that the speech and reasoning capabilities is just obvious to anyone who's been paying attention and has experience with GPT4. The function calling was great, but it had a very simple "world" to interact with. It's really interesting to see it integrated with a robot that can interac…
Figure 01 robot demos its OpenAI integration
81–90 of 116 posts
Re: Figure 01 robot demos its OpenAI integration
#82This shows the real utility of the Groq low latency inference. That delay in responding makes it much less impressive (it’s still really impressive obviously!)
That delay will be eliminated very soon. IMO low latency natural voice conversations are going to be bigger than ChatGPT. It's going to blow people's minds when they can converse with these AIs casually just like with their real life friends. It won't be anything like Siri or Alexa anymore. Here's a demo from a startup in this space. Still very early. https://deck.sindarin.tech/
Re: Figure 01 robot demos its OpenAI integration
#83Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…
Re: Figure 01 robot demos its OpenAI integration
#84Re: Figure 01 robot demos its OpenAI integration
#85Re: Figure 01 robot demos its OpenAI integration
#86Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…
I suspect it's less pre-trained than you think, the tweet is prefaced with a message that all actions are driven by neural networks indicating that it's probably adjusting for the objects, the environment etc.
Re: Figure 01 robot demos its OpenAI integration
#87Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…
I suspect it's less pre-trained than you think, the tweet is prefaced with a message that all actions are driven by neural networks indicating that it's probably adjusting for the objects, the environment etc.
I'd love to be wrong but I expect that if they had that much flexibility in their controllers they would have demonstrated it.
Re: Figure 01 robot demos its OpenAI integration
#88Earlier quoted context omitted.
Can we stop this kind of misinformation? Training a model to map a token to individual letters are no harder than training a model to be fluent at English. Arithmetics with small number of digits are achievable as well. You can just try a small 7B model yourself. If you don't know where to start, try the mistral instruct v0.2, and this is how it goes, > [INST] Spell out the following word letter by letter: margorczyn…
Is Karpathy lying saying that word tokenization brings such problems that can be seen in many LLMs? https://twitter.com/karpathy/status/1657949234535211009 I'm not arguing that you can't use single chars just that many of the issues parent discussed are caused by this.
Re: Figure 01 robot demos its OpenAI integration
#89This is cool. But why would the plate go in the drying rack? It is obviously dirty since there was trash and an apple on it. It should have been washed first.
Re: Figure 01 robot demos its OpenAI integration
#90Earlier quoted context omitted.
> text to servo movement yeah this was super impressive. If this is at the point where you can put an arbitrary object in front of it and ask it to move it somewhere, that's going to be huge for industrial automation type stuff I'd imagine. I do wonder how much of that demo was pre-baked/trained though. Could they repeat the same thing with a banana? What if the table was more cluttered? What if there were two people…
Great question. Boston Dynamics has been demoing the pre-baked dance routine for 2+ decades at this point. Really hoping we can evolve past it.