Even with the two camera approach, you are still doing depth checking with a flat computer vision algorithm. Also keep in mind humans can turn their head, and can handle new situations without training. Humans also can tell situations better that includes dimension and movement.
LiDAR is a physical world depth checking system. It will always beat simulated depth checking. It also does dimension and movement better than computer vision. LiDAR is a 360 degree depth check.
Essentially Tesla is trying to do a LiDAR like point cloud from camera inputs only. That may work in many cases, it will be beat by LiDAR in all cases due to the difference of virtual vs physical data.
> The justification for dropping radar does make sense, says Weinberger, and he adds that the gap between lidar and cameras has narrowed in recent years. Lidar’s big selling point is incredibly accurate depth sensing achieved by bouncing lasers off objects—but vision-based systems can also estimate depth, and their capabilities have improved significantly.
> Weinberger and colleagues made a breakthrough in 2019 by converting camera-based depth estimations into the same kind of 3D point clouds used by lidar, significantly improving accuracy. Karpathy revealed that the company was using such a “pseudo-lidar” technique at the Scaled Machine Learning Conference last year.
> How you estimate depth is important though. One approach compares images from two cameras spaced sufficiently far apart to triangulate the distance to objects. The other is to train AI on huge numbers of images until it learns to pick up depth cues. Weinberger says this is probably the approach Tesla uses because its front facing cameras are too close together for the first technique.
> The benefit of triangulation-based techniques is that measurements are based in physics, much like lidar, says Leaf Jiang, CEO of start-up NODAR, which develops camera-based 3D vision technology based on this approach. Inferring distance is inherently more vulnerable to mistakes in ambiguous situations, he says, for instance, distinguishing an adult at 50 meters from a child at 25 meters. “It tries to figure out distance based on perspective cues or shading cues, or whatnot, and that’s not always reliable,” he says.
> How you sense depth is only part of the problem, though. State-of-the-art machine learning simply recognizes patterns, which means it struggles with novel situations. Unlike a human driver, if it hasn’t encountered a scenario before it has no ability to reason about what to do. “Any AI system has no understanding of what's actually going on,” says Weinberger.
> The logic behind collecting ever more data is that you will capture more of the rare scenarios that could flummox your AI, but there’s a fundamental limit to this approach. “Eventually you have unique cases. And unique cases you can’t train for,” says Weinberger. “The benefits of adding more and more data are diminishing at some point.”
> This is the so-called “long tail problem,” says Marc Pollefeys, a professor at ETH Zurich who has worked on camera-based self-driving, and it presents a major hurdle for going from the kind of driver assistance systems already common in modern cars to truly autonomous vehicles. The underlying technology is similar, he says. But while an automatic braking system designed to augment a driver’s reactions can afford to miss the occasional pedestrian, the margin for error when in complete control of the car is fractions of a percent.
https://spectrum.ieee.org/tesla-places-big-bet-vision-only-s...