Tesla crash in September showed similarities to fatal Mountain View accident

For all those who are clamoring for autopilot to go back into the lab consider this: I have autopilot, and I use it the way people are instructed to. I consider it a companion to my driving, a backup co-driver that has better reflexes. I am in control, and mentally steer the vehicle with my hands on the wheel with the machine. We are driving the car together and I overrule the machine when we disagree. Often the machine catches things before I am able to react. I am 100% convinced that my family is safer with "us" driving them then just me driving them solo.

What do you say to people like me, who, if you took the feature away, would be forced to reduce their safety? It's easy to call for its removal when you only focus on the subset of people who will have their safety increased by removing it (ie, the people, often through no fault of their own, who come to rely upon it more than it is designed for.) What about everyone else that currently benefits from it? Will you force them to go back to driving "solo" to protect those people?

It's not so simple as "Tesla is reckless by releasing this" -- this is providing real safety to people who are using it the way it is intended. Also, the release of autopilot as-is also makes it possible to accelerate the development of full autonomy, which will save lives that otherwise would be lost to accidents if it were to be created later in time. Any analysis which does not account for this dynamic is missing an important piece of the ethics of removing it.

The main question that is worth asking is if there is more Tesla could do to ensure autopilot is used as a co-driver. The name is a poor choice to start.

Even HN readers are misinterpreting it, and there is nothing to prevent confusion from Tesla side on these marketing pages.

The question of purposefulness is mostly irrelevant. It is their responsibility to avoid ambiguity in this domain because this could be dangerous. They are not doing it => they are putting people in danger. Now a posteriori if somebody manages to sue Tesla after a death a bad injury, maybe the purposefulness will be studied (but it will be hard to decide), but a priori it does not matter much as for the consequences of their in any case mislead attempts (even if it was only for an advertising harmless reason in the mind of the people who wrote it like that in the first place).

To finish, that they are carefully choosing their words to be technically true over and over, yet understood in an other way by most of the people, is at least distasteful. That they are doing it over and over through all existing channel makes it more and more probable that this is even on purpose, of course there is no proof but when we reach a certain point, we can be sufficiently convinced without a formal proof (hell even math proofs are rarely checked formally...)

Apparently their radar can only see objects that move.

So, to see this barrier, you'd have to guess its distance from 2D images. That's a hard problem. I think this would be called "obstacle detection using stereo vision" or maybe even "monocular vision" if they're using one camera.

There isn't as much research into this field as there is in systems that use lidar because lidar has been proven to work, and academics probably don't care as much about building their products at massive scales, so the cost of lidar doesn't matter as much.

Searching Arxiv for that phrase [1], the top result is a paper called A Joint 3D-2D based Method for Free Space Detection on Roads [2] that uses Lidar. The next result that doesn't mention lidar is Free Space Estimation using Occupancy Grids and Dynamic Object Detection [3].

Following that is Failing to Learn: Autonomously Identifying Perception Failures for Self-driving Cars [4]. In that paper's conclusion, it states its next plan is to use lidar to improve data quality,

> we plan to incorporate free space computation from the path of the vehicle and from active sensor returns like LIDAR to identify false positives to further improve our assessment and understanding of modern object detectors at the fleet level.

[1] https://arxiv.org/find/all/1/all:+AND+vision+AND+stereo+AND+...

[2] https://arxiv.org/pdf/1711.02144.pdf

[3] https://arxiv.org/pdf/1708.04989.pdf

[4] https://arxiv.org/pdf/1707.00051.pdf

Why not the other way around? You do the driving, and the computer is the backup in case you mess up?

Humans are extremely bad at monitoring monotonic things, even if we are trained to do them. That‘s why trains have elaborate dead man switches to ensure the operator is paying attention.

From these accidents it‘s clear that Teslas aren‘t good enough at making sure the driver is paying attention.

And it‘s also pretty clear that these autopilot victims weren‘t aware how distracted they really were. I‘m pretty sure they thought they had everything under control, right up to the point when their car did something stupid.

Yes, but of course the only thing that matters is whether or not the car can do it. That it requires hardware and software is important to techies but a non-issue to regular drivers. They buy cars, not 'hardware and software'.

And if by some chance it turns out that more hardware was required after all they'll try to shoehorn the functionality into the available package. If only to save some $ but also not to look bad from a PR point of view. That that compromises safety is a given, you can't know today what exactly it will take to produce this feature until you've done so and there is a fair chance that it will in fact require more sensors, a faster CPU, more memory or a special purpose co-processor.