Live data from Hacker News

NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

arstechnica.com

41–50 of 546 posts

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#41
post #3

“[Driver’s] hands were not detected on the steering wheel for the final six seconds prior to the crash. Tesla has said that Huang received warnings to put his hands on the wheel, but according to the NTSB, these warnings came more than 15 minutes before the crash.” This kind of stuff is why I’ve lost all faith in Tesla’s public statements. What they said here was, for all intents and purposes, a flat out lie. Clearly…

> Clearly something went wrong here, but they lept to blaming everyone else instead of working to find the flaw.

Precisely. As I mentioned in another comment: Autopilot indisputably failed.

They are (understandbly) shifting the discussion away from the technical issue (Autopilot failed) to the legal issue (who is to blame), because the latter is something that they can dispute.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#42
NTSB:

• At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph.

• At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle.

• At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle.

• At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increased from 62 to 70.8 mph, with no precrash braking or evasive steering movement detected.

This is the Tesla self-crashing car in action. Remember how it works. It visually recognizes rear ends of cars using a BW camera and Mobileye (at least in early models) vision software. It also recognizes lane lines and tries to center between them. It has a low resolution radar system which ranges moving metallic objects like cars but ignores stationary obstacles. And there are some side-mounted sonars for detecting vehicles a few meters away on the side, which are not relevant here.

The system performed as designed. The white lines of the gore (the painted wedge) leading to this very shallow off ramp become far enough apart that they look like a lane.[1] If the vehicle ever got into the gore area, it would track as if in a lane, right into the crash barrier. It won't stop for the crash barrier, because it doesn't detect stationary obstacles. Here, it sped up, because there was no longer a car ahead. Then it lane-followed right into the crash barrier.

That's the fundamental problem here. These vehicles will run into stationary obstacles at full speed with no warning or emergency braking at all. That is by design. This is not an implementation bug or sensor failure. It follows directly from the decision to ship "Autopilot" with that sensor suite and set of capabilities.

This behavior is alien to human expectations. Humans intuitively expect an anti-collision system to avoid collisions with obstacles. This system does not do that. It only avoids rear-end collisions with other cars. The normal vehicle behavior of slowing down when it approaches the rear of another car trains users to expect that it will do that consistently. But it doesn't really work that way. Cars are special to the vision system.

How did the vehicle get into the gore area? We can only speculate at this point. The paint on the right edge of the gore marking, as seen in Google Maps, is worn near the point of the gore. That may have led the vehicle to track on the left edge of the gore marking, instead of the right. Then it would start centering normally on the wide gore area as if a lane. I expect that the NTSB will have more to say about that later. They may re-drive that area in another similarly equipped Tesla, or run tests on a track.

[1] https://goo.gl/maps/bWs6DGsoFmD2

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#43

self driving systems cant well reason about untrained scenarios or the intent of other humans on the road. I think the people have grossly underestimated how driving in an uncontrolled environment is really a general AI problem, which we're not even close to solving.

Decades away, at which point OEMs will have spent billions on a technology unproven and undesired. They will have bankrupted themselves because there is no viable business model other than SaaS and car companies will not transition to SaaS. They can’t even bring themselves to transition to full-electric platforms.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#44

self driving systems cant well reason about untrained scenarios or the intent of other humans on the road. I think the people have grossly underestimated how driving in an uncontrolled environment is really a general AI problem, which we're not even close to solving.

This wasn't a hard case that needed advanced AI. This was a clearly visible stationary obstacle on a good freeway on a clear California day.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#45
post #7

Earlier quoted context omitted.

Add to that, the last bullet point from page 2 of the official preliminary report[1]: At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increased from 62 to 70.8 mph, with no precrash braking or evasive steering movement detected. That to me is a strong indicator that Tesla's AP2 can not recognize a crashed attenuator, probably one of the strongest argument that…

A counter argument that HUMAN DRIVERS are unsafe an fatal (let alone missing common sense and alertness). I 100% get your point, but my counterpoint, is that this is what we are going to see, in the future. Autonomous systems that by and large are way safer, than human drivers, but when they fail, the might do so, in circumstances where are human would not. Humans are good at some things, computers/ai/sensor suites a…

I shouldn't have to keep saying this in every thread about Tesla autopilot, but I guess I do: the fact that there will probably one day be autonomous driving systems that are safer than human drivers does not imply that Tesla's current system is safer than human drivers.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#46
post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

Tesla cut ties with Mobileye in 2016, and this was a 2017 Model X, so it probably wasn't running Mobileye's software.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#47
Tesla has to realize these "shame the dead dude" posts are PR nightmares, right?

They are reason alone for me to never consider one, that a private moment for my family might end up a pawn in some "convince the public we're safe using any weasel stretch of the facts we can" effort.

If this is disruption, I'll wait for the old guard to catch up, lest I be disrupted into a concrete barrier and my grieving widow fed misleading facts about how it happened.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#48
post #29

Earlier quoted context omitted.

It reads like he was following another car that was driving at a slower speed but once that car left his path, his car began to accelerate to the pre-set speed of 75mph.

It doesn't matter if he was following another car or not. If there is an obstacle, AP2 should brake and in that case it should brake hard or use evasive turning maneuvers to avoid head-on collision.

That’s the thing, it clearly didn’t detect the obstacle.

Given that there were no (known) obstacles, it behaved correctly.

Fix the obstacle detection.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#49
post #7

Earlier quoted context omitted.

Add to that, the last bullet point from page 2 of the official preliminary report[1]: At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increased from 62 to 70.8 mph, with no precrash braking or evasive steering movement detected. That to me is a strong indicator that Tesla's AP2 can not recognize a crashed attenuator, probably one of the strongest argument that…

A counter argument that HUMAN DRIVERS are unsafe an fatal (let alone missing common sense and alertness). I 100% get your point, but my counterpoint, is that this is what we are going to see, in the future. Autonomous systems that by and large are way safer, than human drivers, but when they fail, the might do so, in circumstances where are human would not. Humans are good at some things, computers/ai/sensor suites a…

I get that in 5-10 years we MAY have SDVs that are superior than human drivers. But I think a lot of people have a mental model of SDVs _right now_ being superior to human drivers. I can't remember the crash stats, but I think the only way to know for sure is to calculate the number of miles a single human generally drives on average before they are involved in a collision or crash, etc...

Now I don't have the numbers memorized, but in previous Tesla, Uber, Google threads - only Google had reached records similar to human level crash safety. Keeping in mind however that they are still working on driving through snow, construction sites, non-existant (paint gone) lane lines... so the test areas are quite specific still.

I realize Tesla isn't advertising level 5 autonomy - I'm just responding to the above poster.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#50
post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

>> These vehicles will run into stationary obstacles at full speed with no warning or emergency braking at all. That is by design. This is not an implementation bug or sensor failure.

A little off topic, but I'm curious: I usually use "by design" to mean "an intentional result." How do other people use the term? In this case, the behavior is a result of the design (as opposed to the implementation), but is surely not intentional; I would call it a design flaw.

Post reply on HN