Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Problems in Robotics

scottlocklin.wordpress.com

Post reply on HN
Print view View original

111–120 of 232 posts

Re: Open Problems in Robotics

#111

Post by nmca »

tuatoru wrote: post #47
modeless wrote: post #13

The objections to neural nets as the solution to all the problems on his list are the same as the old objections to neural nets as the solution to computer vision, speech recognition, translation, playing Go, etc. The objections will fall in the face of overwhelming evidence that neural nets simply work better than other approaches to these types of problems. For a long time software was the reason robots didn't work…

> The objections will fall in the face of overwhelming evidence that neural nets simply work better Perhaps? But progress is stalling[1]. We need theoretical foundations. [1] https://www.sciencemag.org/news/2020/05/eye-catching-advance...

Wrg to your reference:

One wonders why the "modest tweaks" weren't used in the original paper - indeed because they weren't known or weren't possible.

The way you should interpet these meta-analysis is "researchers are motivated to attribute improvements in performance to 'the interesting bit' and they will run baselines without applying similar tweaking effort to them"

The progress in the field can in some sense be indicated by the huge improvement in the 2006 method due to the "tweaks" discoverd since.

Re: Open Problems in Robotics

#112

Post by rcxdude »

person_of_color wrote: post #67
paulpauper wrote: post #56

>Multiaxis singularities -this one blew my mind. Imagine you have a robot arm bolted to the ground. You want to teach the stupid thing to paint a car or something. There are actual singularities possible in the equations of motion; and it is more or less an underconstrained problem. I guess there are workarounds for this at this point, but they all have different tradeoffs. It’s as open a problem as motion planning o…

I thought quaternions work around gimbal lock?

In the case of singularities in robotic systems the issue is actually physical (akin to actual gimbal lock in a gimbal with only 3 axes). In certain positions it's not possible to move the end-effector in certain directions, and near those positions you may need large movements of the whole system to make small movements in the end-effector (which can be especially violent if the robot attempts to pass near the singularity at a constant velocity).

The solution, kind of akin to what quternions do, is to have more degrees of freedom than you need (like a 4th ring on a gimbal), but this doesn't eliminate singularities, it just makes them possible to avoid for most of the range of motion (extreme limits may still contain them). Actually coming up with a reasonable solution which avoids them is still not solved in general (though it's solved well enough for practical purposes in a lot of situations).

These singularities exist in humans as well, though we're quite well adapted to avoiding them in normal motions. If you've ever found yourself making an awkward movement (especially needing to readjust your grip to continue a motion), then it's likely you've encountered this without realising it.

Re: Open Problems in Robotics

#113

Post by etrautmann »

treeman79 wrote: post #105
Mvandenbergh wrote: post #98

Earlier quoted context omitted.

Parallax methods are widely used, I think it's how Tesla's driving assist features work. The problem with it is that humans don't just use binocular vision - we have a whole model of the world. So for instance, if I see an object, I usually know roughly how big it is supposed to be because I have a conception of "object". I also know that the straight line on both sides of an object is a wall, and that the wall conti…

Lost binocular vision for a number of years back. You get around just fine without it. Got it back after some speciality glasses (PRISM). Was a complete shock see depth again. Didn’t seem to help anything getting it back. Mostly just trippy. Chairs were amazing to stare at.

I’m curious to hear more. What caused you to lose and then regain depth perception? Were you able to see with both eyes or did you lose vision in one eye temporarily?

Re: Open Problems in Robotics

#114

Post by fxtentacle »

cpgxiii wrote: post #53
fxtentacle wrote: post #44

Earlier quoted context omitted.

I strongly disagree! I have seen fantastic results with stereo cameras, colored lights, and self-calibration. For most use cases, it's no problem if your robot will stop for a few seconds, rotate the camera axis around a bit, and then continue. But that appears to be good enough to calibrate the features for tracking things like a reflective and transparent glass jar. As for the precision, I agree that 1cm at 10m dis…

If you have a vision system that "just works" and produces high-quality pointclouds in an actual kitchen environment, with reflective appliances, silverware, shiny countertops, shiny ceramic dishware, and glasses, we would absolutely use it. We have in-house research work on both learned monocular depth (so-so for robotics tasks) and learned stereo disparity (much more promising), so progress here isn't impossible, b…

It "just works" for our use case. We put in 4K @ 60fps and receive 960x540 @ 20fps of stereo correspondence pairs. So every matched pixel is averaged over 3 frames in time and 4 pixels in every direction in space, meaning 9x9 convolution kernels. In other words, we make the video super clean by area sampling in space and time.

The specific part about our system that makes it usable for me is that for pixels that cannot be matched with a predetermined quality, it'll return a gap marker instead of guessing. For SfM, that means you can just skip those pixels that are affected by reflections moving around. Cooking pots and plates tend to have enough scratches, design, or shape markers to work OK. Wet white floor will usually be flagged as "unknown" except for the grey gaps in between tiles. As for glass, our system can return up to 2 flows per pixel, meaning for a glass mug we get both the mug and a see-through estimate.

If you look at Sintel Clean+"s0-10", you'll find that there are some learned matching algorithms that perform quite well under those conditions: http://sintel.is.tue.mpg.de/quant?metric_id=6&selected_pass=...

We're H-v3 (2nd place) and that 0.284 EPE for s0-10 (slow movements, small disparities) is quite workable, because that means you have on average less than 0.1 pixels of disparity error on the 960x540 depth maps.

As for monocular depth, I see that as mostly of a memorization task again. You train a good stereo matching, then do unsupervised learning on stereo data to get the monocular AI.

I'm at hajo.me and I currently work on stereo matching and depth mapping with the goal of improving VR. Can you disclose which company or what in general you're working on? Also, do you know any discussion groups where optical flow / stereo matching people usually hang out?

Re: Open Problems in Robotics

#115

Post by mrfusion »

I’m excited to see gpt-3 or similar technology applied to these problems.

A robot can now know lots of common sense. Beer is kept in the fridge. The fridge is in the kitchen. Cans should not be shaken, etc, etc. so if something like gpt-3 can help with a high level plan, and regular computer vision can handle the low level stuff like obstacle avoidance, fridge and beer identification, etc

We could have something really interesting.

(Get in touch if you want to work on something like that. I think it would be a blast!)

Edit. What makes me think gpt-3 has some good common sense. I saw someone ask it “can I do a bench press with a cat?” And it said “no, the cat will bite you.” It’s kind of like we’ve achieved what that common sense database project was going for.

Re: Open Problems in Robotics

#116

Post by hnick »

Mvandenbergh wrote: post #98
Autowired wrote: post #96

Earlier quoted context omitted.

Do you think depth estimation could be done with two cameras plus computer vision (to find markers)? I think this is more or less what we do with our own eyes. Of course you would need much more processing power, but maybe for some applications the robot's brain doesn't need to be inside its body.

Parallax methods are widely used, I think it's how Tesla's driving assist features work. The problem with it is that humans don't just use binocular vision - we have a whole model of the world. So for instance, if I see an object, I usually know roughly how big it is supposed to be because I have a conception of "object". I also know that the straight line on both sides of an object is a wall, and that the wall conti…

We can also go up and down abstraction levels at will.

We can see a forest, then a tree, then a leaf. All different "objects", yet related. The ability to not distinguish each blade of grass in my yard and just see it all as "grass" is very useful for noticing and not stepping on what the dog left behind.

Re: Open Problems in Robotics

#117

Post by hwillis »

cpgxiii wrote: post #83
zippy5 wrote: post #74

Earlier quoted context omitted.

Hypothetically, it seems that your problem has to do surfaces being reflective in the visible light spectrum. Do you have any luck with ultrasonic, ultraviolet, or IR sensors? Also I was also thinking about how light source estimation might be viable work around for reflective surfaces. http://www.thomaswhelan.ie/Whelan16ijrr.pdf

Near-IR, which many of the available ToF (Kinect2, K4A, Basler/Lucid/Sony), active stereo (Realsense D4xx, Mynt, plenty more), and many some structured light (all Primesense derivatives, some older Realsense) cameras use, as well as close UV frequencies, suffers much the same issues as visible light. Farther wavelengths in shortwave IR start to have different/better properties, but the costs of cameras are just astro…

> but I'm not aware of any high-resolution ultrasonic sensors for land robots.

Sound cameras are very cool- passive sonar, but they're audio frequency and not usually depth-sensing. There's a lot of room for improvement there, but its a relatively unknown area and definitely hasn't found its niche.

High resolution air sonar is necessarily fairly large, which makes it decidedly unsexy. It also puts a lot of strain on the potential for a low cost BOM when the pcb and chassis are ~>12" wide.

> One minor challenge with ultrasonic sensors in real products is ensuring that they are inaudible to people and pets - when they are almost-audible they are extremely annoying or even painful.

One thing that really surprised me when I got into it was that this isnt a trivial problem. Part of it is practical- the cheapest and most common sensors are around 60-80 kHz (mechanical size makes them easy to produce, with the drawback of a high-ish time constant).

Getting above the range of domestic animals isn't too hard or inconvenient, but it's surprising how extensively evolution has adapted for sound in bats. If you want to avoid harming them you need to keep above ~150 kHz, which gives you a fairly uncomfortably small bandwidth before you're into the highly-attenuating frequency ranges. Underwater it's a different story (biological structures have difficulty resonating at MHz+) but the frequencies that would give you sub-millimeter precision (350 kHz) are way more inconvenient than 40 or 60 kHz. You need totally different types of piezos and the range is sharply limited unless you're projecting volumes that would deafen at audio ranges.

Im still very optimistic about the role of sonar even as cheap lidar and depth cameras crowd the space, but its pretty difficult today to make a truly good sensor.

Re: Open Problems in Robotics

#118

Post by bluGill »

Mvandenbergh wrote: post #98
Autowired wrote: post #96

Earlier quoted context omitted.

Do you think depth estimation could be done with two cameras plus computer vision (to find markers)? I think this is more or less what we do with our own eyes. Of course you would need much more processing power, but maybe for some applications the robot's brain doesn't need to be inside its body.

Parallax methods are widely used, I think it's how Tesla's driving assist features work. The problem with it is that humans don't just use binocular vision - we have a whole model of the world. So for instance, if I see an object, I usually know roughly how big it is supposed to be because I have a conception of "object". I also know that the straight line on both sides of an object is a wall, and that the wall conti…

Note that the human system fails for things like the moon which look vastly different in size at different times. This is proof of your point.

Re: Open Problems in Robotics

#119

Post by bluGill »

NalNezumi wrote: post #87
deepGem wrote: post #77

Earlier quoted context omitted.

"The real problem: solve any of these problems, make very little money" - Just curious why have you come to this conclusion ? Object manipulation has potential products in dishwashing and vegetable chopping - sufficiently large markets, potential billion $ outcomes for a startup which takes the early mover lead. Two robotic hands that can work in co-ordination just as human hands do. Extremely difficult to solve, but…

I'm not the OP, but the real issue in robotics is two-fold and he mention both. 1. The cost & technical limitation. The cost, as some of the solution per problem OP suggest include "Lidar, visions and sensor" and all of those thing cost a lot by themselves. On top of that you need good actuators (Harmonic drives etc) for precision, and then good computing unit to handle all of it. Now you also need more power supply,…

The points you make are all true until they are not. Many useful robots should be able to work well enough for the purpose at a reasonable cost. Part of that is mass production brings prices down.

The problem is of course until we can solve the problem of making it work at all price isn't a consideration. You can go bankrupt advancing the state of the art which then is enough for someone else to take your work and (repeat the bankruptcy part many times until someone works it out), and finally someone makes a ton of money with their useful robot.

Re: Open Problems in Robotics

#120

Post by bluGill »

thdrdt wrote: post #108
contingencies wrote: post #52

Robotics founder here. Popular conceptions of "robots" are unrealistically general. In industry, we do not build robots, we build automation systems. Given the choice, would you prefer an automation system with some environmental assumptions, high speed and perfect repeatability (ie. entire industrial automation world), or no environmental assumptions, crushingly high cost, slow speed and poor reliability (eg. walkin…

Thinking about car painting robots it's clearly about cost. You can program a car painting robot in some hours. This will be way cheaper than a robot that is aware of the car and knows how it should move to perform the best paint job. Another example is your washing machine. It would be nice to have a washing robot that would sort your clothes and wash them. But it is way cheaper to sort clothes yourself. So a cloth…

A robot that is aware of the car is long term better because you don't have to spend hours programming for each car and paint scheme. However the problem is hard enough that right now we spend hours because it works and industrial automation has enough advantages that it is worth doing anyway.
Post reply on HN