Open Problems in Robotics

I work in the perception/localization domain and I am not aware of any large developments in that direction. I do know that there are certain ML based perception systems that got some levels of ASIL certification.

For a long time software was the reason robots didn't work, but that time is ending. Hardware will be the bottleneck soon if it isn't already. No general purpose robot will ever be successful in the mass market using electric motors and gearboxes at each joint. We need simpler, cheaper, lighter, more robust, more reliable, backdrivable, force-sensing actuators.

What is true: The understanding of laymen about what robots can do is very much out of touch with reality. Making real world robots is very hard. I think it's movies which made people think that any 6 year old can just plop together a C-3PO.

What is the hash then? (I omit the points where I lack experience.)

1; Motion planning

"Even things like a model of where the robot is, with respect to the surroundings" That's not motion planing. Motion planing starts from when you have a model of yourself and your surrounding. There are theoretical challenges (imagine a flytrap, or maze) but the real world challenges in my experience come from that under the inaccuracies and glitchyness of perception you are expected to make okay-ish decisions.

3; SLAM

"there’s always going to be new obstacles (a pair of shoes, a book)"

SLAM is about localisation. It gives you a 6dof pose from some fixed coordinate system. It does not deal with tripping hazards. That's obstacle detection. It does indeed deal with "mapping" but only in as much as to get you that pose estimate.

"not turn-key to where there would be a SLAM module you can buy for your robot."

Sure you can. The "Intel Realsense t265" is for example one such a module.

The inside-out tracking of the Oculus Quest is an other (albeit one you will have hard time buying for your robot.)

6; Depth estimation.

That one is odd. True, getting full fidelity depth maps out of monocular images is a research problem. But if in reality you want to estimate the distance to your beer bottle you will use stereo images, or a Kinect like depth sensor. It is obviously a hard engineering challenge to make it four 9 robust, but not an unheard of challenge. If his definition of an "open problem" is that there are people writing research papers about it then it will remain an open problem for a while. But if he just wants to depth estimate stuff, then there are already working good ways. Maybe cost prohibitive, maybe not robust enough for his liking.

9; Scene understanding

Humanity made insane leaps and bounds on this one. Again there are engineering tradeoffs. How much accuracy you get for how much watts. What kind of training data you need, etc etc. Our systems are nowhere as good as a 5 year old human child, but I think we can handle that "beer bottle obscured by ketchup bottle" challenge if we try.

But on the other hand, why not accept the toys on the floor? You are fighting entropy for no reason. You sleep at night, you will work tomorrow during the day, and when you look again, the toys are in an equally dispersed state. Why not let them stay in that state for days until you need to hover?

Your statement that > Motion planning is very much non-convex,

suggests to me that you are very much talking about the same thing. I understand convexity as in a shape. Why is convex good and concave bad in terms of optimization?

I don't want you to dumb down the answer too much as I am a trained Mechanical Engineer but then my major isn't math. Hope you understand :)