Live data from Hacker News

Figure 01 robot demos its OpenAI integration

twitter.com

81–90 of 116 posts

Re: Figure 01 robot demos its OpenAI integration

#81
post #27

It's a really cool demo and I'm impressed by the dexterity of the robot, however I'm a bit underwhelm by what's shown here in the sense that the speech and reasoning capabilities is just obvious to anyone who's been paying attention and has experience with GPT4. The function calling was great, but it had a very simple "world" to interact with. It's really interesting to see it integrated with a robot that can interac…

[deleted]

Re: Figure 01 robot demos its OpenAI integration

#82

This shows the real utility of the Groq low latency inference. That delay in responding makes it much less impressive (it’s still really impressive obviously!)

That delay will be eliminated very soon. IMO low latency natural voice conversations are going to be bigger than ChatGPT. It's going to blow people's minds when they can converse with these AIs casually just like with their real life friends. It won't be anything like Siri or Alexa anymore. Here's a demo from a startup in this space. Still very early. https://deck.sindarin.tech/

demod this. The ability to interrupt the language model is very cool, However, i notice, it failed to move onto the next slide often. It could never get to the final slide without explicit mention to go there, and when i got to the last slide, i asked to go back to the first slide, it would say "ok lets go to the last slide" everytime, these are probably more control issues than language model issues but i thought id point them out, just in case.

Re: Figure 01 robot demos its OpenAI integration

#83

Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…

I suspect it's less pre-trained than you think, the tweet is prefaced with a message that all actions are driven by neural networks indicating that it's probably adjusting for the objects, the environment etc.

Re: Figure 01 robot demos its OpenAI integration

#84
The most impressive part of this demo, to me, is the robot "seeing" and picking up objects with human-like appendages. I must have missed something, but I was under the impression that this was very hard. As I understand it inverse kinematics is pretty hard - did they solve it with NNs?

Re: Figure 01 robot demos its OpenAI integration

#86

Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…

I suspect it's less pre-trained than you think, the tweet is prefaced with a message that all actions are driven by neural networks indicating that it's probably adjusting for the objects, the environment etc.

It is safe to assume it works only for that specific demo, and nothing else.

Re: Figure 01 robot demos its OpenAI integration

#87

Selecting one of a set of pre-trained actions by voice is cool but not exactly ground-breaking. Using GPT-4V to describe a scene is also pretty simple. The most impressive things here to me are the speed of picking up the trash and the fluid passing of objects between hands. It's unclear how general these movement policies are though. The way that guy is standing perfectly still makes me think that it would fail if e…

I suspect it's less pre-trained than you think, the tweet is prefaced with a message that all actions are driven by neural networks indicating that it's probably adjusting for the objects, the environment etc.

I think it's true that it can adjust, somewhat. I expect that it could handle slight variations such as moving any of the items on the table around by a few inches, or adding or removing a couple of plates from the rack. However, I do not expect that it could handle larger variations like replacing the apple with a pineapple, picking trash out of the dish rack or the cup instead of the plate, replacing the cup with a coffee mug, or sliding the dish rack over to the other side of the robot.

I'd love to be wrong but I expect that if they had that much flexibility in their controllers they would have demonstrated it.

Re: Figure 01 robot demos its OpenAI integration

#88
post #53

Earlier quoted context omitted.

Can we stop this kind of misinformation? Training a model to map a token to individual letters are no harder than training a model to be fluent at English. Arithmetics with small number of digits are achievable as well. You can just try a small 7B model yourself. If you don't know where to start, try the mistral instruct v0.2, and this is how it goes, > [INST] Spell out the following word letter by letter: margorczyn…

Is Karpathy lying saying that word tokenization brings such problems that can be seen in many LLMs? https://twitter.com/karpathy/status/1657949234535211009 I'm not arguing that you can't use single chars just that many of the issues parent discussed are caused by this.

Letter-by-letter tokenization increases inference and training costs and latency (as you need more tokens)

Re: Figure 01 robot demos its OpenAI integration

#89
post #54

This is cool. But why would the plate go in the drying rack? It is obviously dirty since there was trash and an apple on it. It should have been washed first.

Because the robot likely doesn't have any memory. It sees empty plate after the trash got removed.

Re: Figure 01 robot demos its OpenAI integration

#90
post #16
post #5

Earlier quoted context omitted.

> text to servo movement yeah this was super impressive. If this is at the point where you can put an arbitrary object in front of it and ask it to move it somewhere, that's going to be huge for industrial automation type stuff I'd imagine. I do wonder how much of that demo was pre-baked/trained though. Could they repeat the same thing with a banana? What if the table was more cluttered? What if there were two people…

Great question. Boston Dynamics has been demoing the pre-baked dance routine for 2+ decades at this point. Really hoping we can evolve past it.

Not true. Boston Dynamics has been demoing long and complicated single take videos involving walking, running, jumping and manipulation, in less than ideal conditions.
Post reply on HN