That's very cool. I have almost no experience with robotics, so excuse the silly questions: - How does it know what objects are? Does it use some sort of realtime object classifier neural net? What limitations are there here? - Does the robot know when it can't perform a request? I.e. if you ask it to move a large box or very heavy kettlebell? - How well does it do if the object is hidden or obscured? Does it go look…
Disclaimer: I'm not one of the authors, but I work in this area. You basically hit the nail on the head with these questions. This work is super cool, but you named a lot of the limitations with contemporary robot learning systems. 1. It's using an object classifier. It's described here ( https://github.com/ok-robot/ok-robot/tree/main/ok-robot-navi... ), but if I understanding it correctly basically they are using a…
Will all the recent work to make gen. AI faster (see groq for LLM & fal.ai for stable diffusion), I wonder if the latency will become low enough to make this a non-issue or at least good enough