Tesla’s Push to Build a Self-Driving Car Sparks Dissent Among Its Engineers

Aerospace is the /absolute/ leader for driver assistance. They have decades of experience, especially in the field of ergonomics and brilliantly crafted semi-automated procedures[1].

But in the end, we all accept the risk of riding planes that we don't control because we entrust our lives to /trained pilots/ not because of such systems. As an illustration, the debate is still vigorous about whether or not a computer should be allowed to sit betwen the driver and the actuators [2]. It is also the case for cars, especially after the Toyota blunders [3] so I do believe all this body of experience is relevant and cannot be easily "disrupted".

[1] https://en.wikipedia.org/wiki/Traffic_collision_avoidance_sy... [2] https://aviation.stackexchange.com/questions/149/what-are-th... [3] http://www.edn.com/design/automotive/4423428/Toyota-s-killer...

> If the computer safely drives itself 99% of the time but in that 1% when the human needs to take control, the human fucks up, the occupants of the vehicle are still dead

Not dead, which I feel is important to point out. Involved in an incident, possibly a collision or loss of lane, but really it's quite hard to get dead in modern cars. A quick and dirty google shows 30,000 deaths and five and a half million crashes annually in the US - that's half a percent.

So in your hypothetical the computer drives 99% of the time, and of the 1% fuckups, less than 1% are fatal.

Well, for one you're passing your incompetence off to other drivers to deal with, something that will inevitably lead to accidents behind the car that slows down without any actual reason, for another because driving is a lot more complex than flying when it comes to automation. You might expect the opposite but pilots routinely describe their careers as 30 years of boredom punctuated by 30 seconds of sheer panic.

This is a good summary of modern risk culture. It has good sides and bad sides. On the good side, you have things like commercial aircraft and trains that, despite them being inherently dangerous, manage to achieve extremely low fatality rates.

On the bad side, this risk culture destroys large scale innovation, and even safety in the long run. The problem is that we've adopted an attitude that safety is always first, meaning that it is immoral to do something in a less safe way, no matter what the other benefits might be. This means we get a regulatory, tort, and engineering culture that is willing to use existing systems, because they are grandfathered and therefore "reasonable", but will only adopt new systems if they can be shown to be perfectly safe.

This culture is fairly new. I date it to sometime in the seventies. Ralph Nader and the Pinto were both symptoms and causes. You can see the transition in, for example, how America responded to the Apollo 1 fire vs. the Challenger accident.

Since you're a a rail engineer, let's look at rail mass transit systems. The NYC subway has a limit of roughly 30 trains per hour, a two minute headways. All of the braking rates, margins of safety, and signaling systems do their job, and you never see two trains hitting each other. During rush hour though, trains are packed way over capacity, and this is mostly because of this headway limitation. If you were to imagine this on the freeway, you'd have to leave two miles between you and the next car. The cost for this level of safety is that about half a million people have to spend an hour every day in miserable conditions. Many of them choose to drive or take cabs instead, to avoid this.

When they make this decision, none of them even give the slightest thought that driving is maybe 10x more dangerous than taking the train. They understand that safety for both is plenty good, and the way that they spend two hours of their day, 13% of their waking hours, is a lot more important than some tiny difference in their risk of dying on the way to work.

Ultimately, while very well intentioned, this safety culture is inhuman. It's pessimistic. It says "nothing in your life could possibly be so important that it's worth any possibility of you being injured. "

So, this is what we face now, over and over. With self driving cars, it's, "sure, the chance of you getting killed is half as much as if you had driven, plus you get all of that time back to have a nice conversation, read a book, or idly stare out the window, but since it doesn't meet our aviation/rail transit level of safety, you can't have it." You even see it with kids -- your twelve year old kids can't be allowed out of the sight of an adult, or you're a dangerous, neglectful parent. It does not matter that childhood is the process of learning how to be an adult, and that becoming an adult is a process of progressively mastering greater and greater freedoms -- the most important thing is that we are never seen to expose children to even the most minute level of risk.

I really want us to have a real conversation about acceptable risk, informed consent, and human progress. We need it badly to regain our souls.

You confuse technological advances and safety techniques. These are very different things. Safety techniques are theoretical principles that are used to keep technology safe. Typical examples of safety techniques in systems design are: readback, diversity, majority voting, coding of information.

In the case of establishing the correct position of vehicles, whatever the technology used it will have an error of measurement. This error accumulates over time, especially if you are measuring displacement. You will need reference points to retain your positional uncertainty within an acceptable value. An autonomous vehicle uses a map containing features that can be used as reference points and triangulate its position based on that. A human brain does this constantly using its eyes, ears and memory. It is an example of diversity: you correlate the perceived displacement by your internal ear with what your eyes sees, using a reference learned in your brain about your surroundings and sensors' ability.

To reach 1E-9, we cannot rely on such self-learned things: we can agree that the probability for the landscape to change is quite higher than 1E-9. In the case of trains, driverless trains use coded beacons / loops or GPS diversity. Such techniques imply that the infrastructure around your vehicle collaborates to the safety of the system. Hence my statement.

This being said, the rail industry has been dreaming for long of a completely vehicle centric solution. There are high stakes for that: reduced costs, competitive advantage, etc. Last attempt here: http://www.alstom.com/products-services/product-catalogue/ra...