Tesla crash in September showed similarities to fatal Mountain View accident • Hacker News Board

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#531

Post by dragonwriter » Thu, Apr 05, 2018, 7:35 PM UTC

pjkundert wrote: post #530

The root problem (as described by Taleb in "Skin In The Game", is: not all statistical probabilities are "ergodic" -- have no risk of Ruin or Extinction. The problem with saying that "our automatic driving system is much better than the average driver" and thus you should be good with allowing our system to drive, is that you are not exposed -- serially -- to the whims of some average driver. One "average driver" mis…

> So; since automatic driving systems are non-ergodic, they must be dramatically better (ie. orders of magnitude better) than the "average driver" to be considered viable.

The risks from human drivers are also non-ergodic, so that suggests that if automatic driving systems are even minimally better than the average driver you should demand that every other driver be replaced with them without delay, because you will die sooner with human drivers on the road.

Of course, you won't want to be replaced unless the automatic driver is better than you, and human drivers all think they are far above average. And that’s the real problem.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#532

Post by pjkundert » Thu, Apr 05, 2018, 8:02 PM UTC

dragonwriter wrote: post #531

pjkundert wrote: post #530

The root problem (as described by Taleb in "Skin In The Game", is: not all statistical probabilities are "ergodic" -- have no risk of Ruin or Extinction. The problem with saying that "our automatic driving system is much better than the average driver" and thus you should be good with allowing our system to drive, is that you are not exposed -- serially -- to the whims of some average driver. One "average driver" mis…

> So; since automatic driving systems are non-ergodic, they must be dramatically better (ie. orders of magnitude better) than the "average driver" to be considered viable. The risks from human drivers are also non-ergodic, so that suggests that if automatic driving systems are even minimally better than the average driver you should demand that every other driver be replaced with them without delay, because you will…

The real problem is that, yes, human drivers ARE better than computers — for some unknown subset of driving. It’s that small subset of lethal mistakes you avoid by not letting your computer drive that makes all the difference.

The computer must not just be better on average — people must know that it’s better overall, across all possible weird events that a human just might throw a Hail-Mary and survive!

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#533

Post by Animats » Thu, Apr 05, 2018, 8:11 PM UTC

Piskvorrr wrote: post #330

lozenge wrote: post #166

Earlier quoted context omitted.

They had a falling out with their software supplier Mobileye which set them back a year or more.

Did I just read "they found out that tech is not actually ready to fulfill what its marketing promised", or is that something else?

Two years ago, Tesla claimed self-driving in two years. Last December, Musk was claiming they need 3 more years.[1]

Waymo is currently the only credible player in self-driving. Uber crashed. Volvo backed off. Tesla has problems. Cruise just had one of their cars pulled over and ticketed in SF.[2] Waymo's cautious approach is paying off.

[1] https://electrek.co/2017/12/08/elon-musk-tesla-self-driving-... [2] http://sanfrancisco.cbslocal.com/2018/03/27/self-driving-car...

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#534

Post by AnimalMuppet » Thu, Apr 05, 2018, 8:44 PM UTC

pjkundert wrote: post #532

dragonwriter wrote: post #531

Earlier quoted context omitted.

> So; since automatic driving systems are non-ergodic, they must be dramatically better (ie. orders of magnitude better) than the "average driver" to be considered viable. The risks from human drivers are also non-ergodic, so that suggests that if automatic driving systems are even minimally better than the average driver you should demand that every other driver be replaced with them without delay, because you will…

The real problem is that, yes, human drivers ARE better than computers — for some unknown subset of driving. It’s that small subset of lethal mistakes you avoid by not letting your computer drive that makes all the difference. The computer must not just be better on average — people must know that it’s better overall, across all possible weird events that a human just might throw a Hail-Mary and survive!

> It’s that small subset of lethal mistakes you avoid by not letting your computer drive that makes all the difference.

The small subset of lethal mistakes you avoid by not letting yourself drive matter too...

Here's what I think is really going on. There's a transition from "not good enough to trust with a human life" to "approximately as likely as a human to make a fatal mistake, even if in different ways" to "clearly less likely to cause a fatal accident than even a good human driver". But there's also a huge pot of money for whoever wins this market. So at least some players are pretending that they're further along the transition to "better than human", hoping that the market believes them.

A second point: when we're in the state of "approximately as likely as a human to make a fatal mistake, even if in different ways", the humans look at the ways that the computer makes, and think: "That's a really stupid mistake to make. How could it be so dumb?" But just because the mistakes aren't ones that a human would make very often, that doesn't mean that the computer isn't on par with humans in terms of overall death rate. But the computer still looks bad on the mistakes it makes.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#535

Post by mdorazio » Thu, Apr 05, 2018, 9:04 PM UTC

Dylan16807 wrote: post #85

mdorazio wrote: post #74

Earlier quoted context omitted.

It's mostly due to limited sensors and immature algorithms. If you think about a sharp left turn with a wall on the right side of the road, there is a "stationary object" (the wall) in front of the vehicle throughout the turn that will be detected by radar and cameras. In order to navigate a situation like this, an autonomous vehicle has to have additional decision-making in place to effectively override "stationary…

> But once you do that, it gets tricky to determine when you should ignore the "stationary object" and when you shouldn't. That's not tricky. You never ignore it. A stationary object means that you will have to either stop or turn. You don't know which one yet, but you definitely know you'll have to take an action. If you haven't decided by some close point, you just pick one.

"Ignore" was probably the wrong word, but it is tricky. Remember that a vehicle is evaluating the environmental state at a given point in time to make decisions. Let's take the example of everyday driving on the freeway where you are following a car going the same speed as you. From your car's radar and camera perspective, the vehicle in front of you is stationary. It's not until you combine that information with additional information (your own velocity) and infer a more complicated state of affairs that you get a more complete picture. The camera and radar are both screaming "there's a stationary object right in front of us!", but you're still traveling at 60mph directly toward it.

Combine that with the normal event of a garage wall or building directly in front of you while parking and we have three different "stationary object in front" situations that all have to be handled differently. In all three, the radar and camera are saying "stationary object ahead". But in the left-turn-with-wall situation you need to turn instead of stop. In the following-another-car situation you do nothing, and in the parking situation you need to stop instead of turn. And that's all without even starting to account for things like rain (opaque to some LIDAR and cameras = "we're literally crashing right now!" while radar reports nothing amiss). In order to decide what to do you have to pull in additional environment information, but then that's going to have similar caveats and conflicting reports to account for, so you need additional rules and information, etc. and now you've snowballed into a highly complex model with corner cases and potential bugs everywhere.

If it was easy we would have had self-driving cars a decade ago.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#536

Post by ksk » Thu, Apr 05, 2018, 9:10 PM UTC

ajross wrote: post #529

matte_black wrote: post #442

Earlier quoted context omitted.

Drunk drivers, recklessly fast drivers, redlight runners, stop sign blowers, high speed tailgaters... those are the demographic you have to compare the Tesla drivers to. Do people who buy Teslas engage in those kinds of dangerous activities?

> Drunk drivers, recklessly fast drivers, redlight runners, stop sign blowers, high speed tailgaters. Wait, so you are willing to share the road with all those nutjobs, yet you're "staying the hell away from" Teslas you see which you claim are NOT being driven by these people? I think you need to reevaluate your argument. Badly. That even leaves aside the clear point that a Tesla on autopilot is significantly less li…

>That even leaves aside the clear point that a Tesla on autopilot is significantly less likely to make any of those mistakes...

What are you basing this on? Specifically how is it 'clear' and what data has shown this to be 'significant' ?

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#537

Post by ksk » Thu, Apr 05, 2018, 9:13 PM UTC

stale2002 wrote: post #500

Ajedi32 wrote: post #393

Earlier quoted context omitted.

I think the point he's making is that it's worse if a person dies because of something they have no control over (self-driving car malfunction) than if a person dies because of their own stupid choices (driving drunk, driving too fast for conditions, running red lights, etc). This, of course, ignores the fact that stupid choices drivers make tend to affect other people on the road who did nothing wrong, so the introd…

> I think the point he's making is that it's worse if a person dies because of something they have no control over Perhaps. I reject this argument though. A death is a death. And if you prevent lifesaving technology then you are responsible for a whole lot of deaths, irregardless of how "deserved" those deaths are.

>Perhaps. I reject this argument though. A death is a death.

A death caused by shitty engineering (e.g. Tesla in this case) is not the same as a death caused by gross negligence on ones own part.

>And if you prevent lifesaving technology then you are responsible for a whole lot of deaths, irregardless of how "deserved" those deaths are.

How many deaths has Tesla prevented, which would have happened otherwise?

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#538

Post by ksk » Thu, Apr 05, 2018, 9:21 PM UTC

ajross wrote: post #380

antirez wrote: post #200

What is happening was clear to many since the start: Tesla incarnates the behavior of his founder, exaggerating the technology that is really available right now, and selling it as a product without the premise for it to be safe. A product that kinda drives your car but sometimes fails, and requires you to pay attention, is simply a crash that we are waiting to happen. And don't get trapped by the "data driven" analy…

> 1. Ethical. One thing is that you do a stupid thing and die. Another is that a technology fails in trivial circumstances that in theory are avoidable. A doctor can make also errors, but a medical device is required to be very safe and reliable in what it does. This needs elaboration or citation. You're one of the only folks I've seen come straight out and make this point: that somehow a technology or practice that…

>In fact, medical devices seem actually counter to your argument. No one says an automatic defibrillator's decisions as to when to fire are going to be as good as a human doctor, and in fact these things have non-zero (often fatal) false positive rates. But they save lives, so we use them anyway.

Yes, we would also tolerate Teslas if they were critical life support technology. How many lives has it saved BTW?

>and in fact these things have non-zero (often fatal) false positive rate

What is non zero? Ten to the power -20?

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#539

Post by ggg9990 » Thu, Apr 05, 2018, 9:22 PM UTC

jacquesm wrote: post #467

whichdan wrote: post #453

Earlier quoted context omitted.

Well, this is pretty classist...

The reason why people buy expensive cars is because it is one of the ways in which they can quickly and quietly signal wealth or status. Cars are classicist, like it or not. After all, how much better can a $500K supercar be compared to a $50K car? Definitely not ten times better, the speed limits are the same, seating capacity is likely smaller, there may be a marginal improvement in acceleration and a corresponding…

Some of us just really like nice cars without signaling wealth. I recently bought a Hyundai Genesis even though I liked and could afford the better Mercedes Benz because I preferred to have a non-luxury brand. It's a New England thing.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#540

Post by matte_black » Thu, Apr 05, 2018, 9:28 PM UTC

ksk wrote: post #536

ajross wrote: post #529

Earlier quoted context omitted.

> Drunk drivers, recklessly fast drivers, redlight runners, stop sign blowers, high speed tailgaters. Wait, so you are willing to share the road with all those nutjobs, yet you're "staying the hell away from" Teslas you see which you claim are NOT being driven by these people? I think you need to reevaluate your argument. Badly. That even leaves aside the clear point that a Tesla on autopilot is significantly less li…

>That even leaves aside the clear point that a Tesla on autopilot is significantly less likely to make any of those mistakes... What are you basing this on? Specifically how is it 'clear' and what data has shown this to be 'significant' ?

Elon Musk probably