NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash • Hacker News Board

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#131

Post by jedberg » Thu, Jun 07, 2018, 5:53 PM UTC

curiousgal wrote: post #39

jackson1way wrote: post #30

Despite the autopilot failure, I find the battery failure quite remarkable too: > The car was towed to an impound lot, but the vehicle's batteries weren't finished burning. A few hours after the crash, "the Tesla battery emanated smoke and audible venting." Five days later, the smoldering battery reignited, requiring another visit from the fire department. Where is your LiPo god now? Batteries have more energy densit…

Battery failure? You're expecting their engineers to design a battery that remains safe after a 70mph crash into a barrier?

ICE makers have to design gas tanks that are safe after a 70mph collision, so yeah, it's kind of expected that the battery is safe too.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#132

Post by Someone1234 » Thu, Jun 07, 2018, 5:53 PM UTC

maratd wrote: post #119

tptacek wrote: post #75

Earlier quoted context omitted.

Pedantic: Mobileye cut ties with Tesla, not the other way around.

Yes, after Mobileye (now Intel) learned there was an internal Tesla effort to replace them.

Isn't that claim also from Tesla?

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#133

Post by CydeWeys » Thu, Jun 07, 2018, 5:55 PM UTC

jakelarkin wrote: post #129

CydeWeys wrote: post #40

Earlier quoted context omitted.

Disagree. I think that the likes of Waymo and Cruise understand exactly how hard of a problem this is. It's only companies like Uber and Tesla that are underestimating it, and doing stupid things like disabling automatic braking (Uber) or not using LIDAR (Tesla).

the best software engineering company in the world has spent over a decade and hundreds of $millions on this problem and they are still stuck in a private beta in the easiest road environment they could find. what does that tell you about the feasibility of the solution. There's no evidence that Cruise is in the same class as Waymo.

Waymo is about to start ferrying around paying customers later this year. Nobody said the problem isn't hard, but it definitely does not seem infeasible.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#134

Post by GeekyBear » Thu, Jun 07, 2018, 5:55 PM UTC

tptacek wrote: post #75

binarybits wrote: post #46

Earlier quoted context omitted.

Tesla cut ties with Mobileye in 2016, and this was a 2017 Model X, so it probably wasn't running Mobileye's software.

Pedantic: Mobileye cut ties with Tesla, not the other way around.

More relevant in this situation: Mobileye's stated reason for cutting those ties.

>On Wednesday, Mobileye revealed that it ended its relationship with Tesla because "it was pushing the envelope in terms of safety." Mobileye's CTO and co-founder Amnon Shashua told Reuters that the electric vehicle maker was using his company's machine vision sensor system in applications for which it had not been designed.

"No matter how you spin it, (Autopilot) is not designed for that. It is a driver assistance system and not a driverless system," Shashua said.

https://arstechnica.com/cars/2016/09/tesla-dropped-by-mobile...

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#135

Post by thatswrong0 » Thu, Jun 07, 2018, 5:55 PM UTC

jumelles wrote: post #77

curiousgal wrote: post #39

Earlier quoted context omitted.

Battery failure? You're expecting their engineers to design a battery that remains safe after a 70mph crash into a barrier?

Yes? Welcome to the automotive industry. Lives are at stake.

Is there an equivalent safety standard for ICE vehicles? I don't think a gas task would do particularly well in the exact same circumstance either

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#136

Post by wyldfire » Thu, Jun 07, 2018, 5:56 PM UTC

nlawalker wrote: post #50

Animats wrote: post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

>> These vehicles will run into stationary obstacles at full speed with no warning or emergency braking at all. That is by design. This is not an implementation bug or sensor failure. A little off topic, but I'm curious: I usually use "by design" to mean "an intentional result." How do other people use the term? In this case, the behavior is a result of the design (as opposed to the implementation), but is surely not…

> but is surely not intentional; I would call it a design flaw.

In fact, it is intentional! Meaning that the system has a performance specification that permits failure-to-recognize-lanes-correctly in some cases. This element of the design relies on the human operator to resolve. Once the human recognizes the problem, either they disengage the autopilot or engage the brakes/overpower the steering.

Now, you could argue that the design should be improved and I would agree. But we should perhaps step back and consider some meta-problems here. As others have stated, the functionality cannot deviate significantly from previous expectations without at a bare minimum an operator alert, training pamphlet or disclaimer form. Tesla's design verification likely should be augmented to more comprehensively test real-world scenarios like this.

But the real core issue is that the design approaches this uncanny valley of performing so terribly close to parity with human drivers that human drivers let their guard down. IMO it's the same problem as the fatality in Phoenix w/an Uber safety driver (human driver). When GOOG's self driving program first monitored their safety drivers they found that they didn't pay attention, or slept in the car. IIRC they added a second safety driver to try to mitigate that problem.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#137

Post by slg » Thu, Jun 07, 2018, 5:57 PM UTC

txcwpalpha wrote: post #117

falcolas wrote: post #97

Earlier quoted context omitted.

One more thing to note - anecdotal evidence indicates that Tesla cars did not attempt to center within a lane prior to an OTA update, after which multiple cars exhibited this "centering" action into gore sections (and thus required manual input to avoid an incident) on video. To me, that this behavior was added via an update makes it even harder to predict - your car can pass a particular section of road without inci…

And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal flying at 70mph with humans inside…

This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and did a voluntary recall. That means many of those cars with buggy brake systems were on the road for years after a potential fix was available and were driven for billions of potentially unsafe miles.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#138

Post by chinhodado » Thu, Jun 07, 2018, 5:57 PM UTC

rcxdude wrote: post #73

baconmania wrote: post #61

Earlier quoted context omitted.

What's your source for the claim that Tesla's system "doesn't detect stationary objects"? From the reference frame of a moving Tesla, both globally stationary and globally moving objects will appear to be in motion.

Yes, but the radar has poor angular resolution (but a good idea of relative velocity), so it cannot tell the difference between a stationary object at the side of the road and one in the middle of it, so it must ignore all stationary object (by ignoring all objects with an apparent velocity approximately equal to the speed of the vehicle) in order to avoid constant false positives.

Can you explain further what "poor angular resolution" means?

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#139

Post by metalliqaz » Thu, Jun 07, 2018, 5:57 PM UTC

Animats wrote: post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

But why would it speed up beyond the speed limit?

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#140

Post by jungturk » Thu, Jun 07, 2018, 5:58 PM UTC

Analemma_ wrote: post #91

jungturk wrote: post #64

Earlier quoted context omitted.

Not sure about Elon, but I suspect (hope?) most of us Tesla drivers have familiarized themselves with the limitations of the system. Its got a couple scenarios in my experience where it shines - chiefly long drives on clean clear interstates and stop-and-go traffic - but its got suicidal tendencies in others. Its not unlike being driven by a tipsy friend or a teenager - there are enough indications that it can't be t…

To be blunt, I'm kind of stunned that you are defending a system that you yourself describe as "like being driven by a tipsy friend". What?! Sure, I could sit in the front passenger seat while being driven by my tipsy friend and grab the wheel from him if he was about to steer into a wall, but that is clearly much less safe than just driving myself. And I definitely wouldn't pay for the privilege nor endorse it as a…

Its curious to see you appeal to ad hominems, outrage, and assumptions of bad intent, but I'll give it a go anyhow.

Perhaps I wasn't clear enough about the two use cases I cited, but the car performs excellently (and entirely soberly) in those situations.

It's a tool that has significant utility and significant footguns. I'm not advocating whether you should use it or not - I'm sharing my experience with where the system delivers and hoping users are looking past the hyperbolic marketing.

Its abundantly clear (even to us in the Elon personality cult?) that Tesla and Elon have done themselves a disservice with how they handled this tragedy.