Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

arstechnica.com

Post reply on HN
Print view View original

321–330 of 546 posts

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#321

Post by kchhina »

27182818284 wrote: post #13

The NTSB report confirms that. The crash attenuator—an accordion-like barrier that's supposed to cushion a vehicle when it crashes into the lane separator—had been damaged the previous week when a Toyota Prius crashed at the same location. The resulting damage made the attenuator ineffective and likely contributed to Huang's death. kinda sounds like maybe that part of the road isn't well designed or marked too.

It should be fairly easy to detect a chevron sign with yellow and black stripes [0] and take evasive measures to slow down, brake heavily or change lanes. From my understanding of CNN neural networks it's a fairly simple classification problem (that A. Karpathy researched, and taught at Stanford; and he's now leading the AI/vision team at Tesla).

Looking at the video of the guy that tried that same route again, it looks like the sun was straight ahead (see starting 37th second of video [1]), and I am wondering if the camera had an issue identifying the sign due to the glare from the sun.

Do we know if this problem has been fixed now? Can the chevron be identified at the same location by other Tesla drivers today, months after the guy that tested it and posted the video of his car swerving at the same place [1]?

Following lane markings are never going to be a 100% reliable method for navigating roads. Even humans miss merging lanes, or have issues at complex surface street intersections where there are multiple overlapping lanes. But humans take evasive action.. they slow down, re-access, look at signs and probably get honked at.

Following other cars is never going to be a reliable method for navigating roads. I don't know what the guy in front of me is going to do! I keep my safe distance and take evasive action if the guy in front of me does something stupid.

I feel signs are most reliable in that sense. If there is a missing sign, a human is more likely bound to make a mistake as well. Maybe I am biased, because as someone who sails, signs (buoys, lights, colors, etc.) are what guide me in my navigation as there are no lane line markings on the water (not a perfect analogy because the speeds/traffic patterns are very different, but the point I am trying to make is about humans identifying the signs with muscle memory). A vision based neural network should be able to do the same.

[0] https://english.stackexchange.com/questions/249780/what-do-y... [1] https://youtu.be/VVJSjeHDvfY?t=37

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#322

Post by mirimir »

Animats wrote: post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

Shortly after the accident, a few other Tesla drivers reported similar behavior approaching that exit.

But on the other hand, gores like that can also trick human drivers. Especially if tired, with poor visibility. In heavy rain or whiteout, I typically end up following taillights. But slowly.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#323

Post by jonhendry18 »

JackCh wrote: post #182
mmusson wrote: post #116

Earlier quoted context omitted.

I wonder how soon radar reflecting paints will start being used on roadways and barriers, etc. That seems like it would be a general benefit to all brands of cars with auto drive. At the end of the day we want to know if the autonomous systems are safe. Policy decisions will end up depending on that determination. That requires clear definitions for what constitutes failures and accurate gathering of data.

These cars aren't crashing into stationary objects that don't have a radar return. The software in the cars is deliberately filtering out the radar returns of stationary objects. It wouldn't matter how many radar reflectors you slapped on the barrier, the Tesla would ignore the radar return of the stationary barrier.

Maybe the barrier needs a big waving maneki-neko cat in front of it, if the waving paw would be enough movement to be noticed by the radar.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#324

Post by Robotbeat »

abalone wrote: post #79

> During the 18-minute 55-second segment, the vehicle provided two visual alerts and one auditory alert for the driver to place his hands on the steering wheel. These alerts were made more than 15 minutes prior to the crash. Whoah. So there were NO alerts for 15 minutes prior to the crash. Compare this with Tesla's earlier statement: > The driver had received several visual and one audible hands-on warning earlier in…

It sounds like exactly the same thing to me.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#325

Post by Robotbeat »

mcguire wrote: post #297

Here's the most interesting quote to me: " The crash created a big battery fire that destroyed the front of Huang's vehicle. "The Mountain View Fire Department applied approximately 200 gallons of water and foam" over a 10-minute period to put out the fire, the NTSB reported. " The car was towed to an impound lot, but the vehicle's batteries weren't finished burning. A few hours after the crash, "the Tesla battery em…

It's a lithium-ion battery, so it needs to be fully discharged.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#326

Post by Robotbeat »

prawn wrote: post #294
kevincox wrote: post #205

Earlier quoted context omitted.

> Don’t want to see autopilot disabled as a result of this I would. I think any self-driving system that expects the "driver" (passenger I guess) to take over in tough situations is bound to fail again and again. If people feel like they don't need to pay attention they won't no matter how much they know that they should. I think that partial autonomy is a sort of uncanny valley, either side is much safer.

I'd prevent them from using the AutoPilot term, penalise them for marketing imagery that includes people driving with their hands off the wheel, and require hands to be on the wheel far more often. Until it's significantly improved, it should be a backup safety system rather than a driving and safety system.

>I'd prevent them from using the AutoPilot term

I disagree. It's the same term as in aviation and does the same thing.

> penalise them for marketing imagery that includes people driving with their hands off the wheel

That's more reasonable.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#327

Post by dchichkov »

akira2501 wrote: post #315
Pfhreak wrote: post #299

Earlier quoted context omitted.

This seems testable. IANAAE (I Am Not An Automotive Engineer), but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Like, if in one case the new code would want you to move left, and the old code goes straight, one of those behaviors is probably wrong. If the driver corrects, then the new code is probably correct, but if the driver doe…

> but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Because there are no closed facilities that you can use to actually perform any meaningful test. You could test "in-situ", but you would need an absolutely _huge_ testing area in order to accurately test and check all the different roadway configurations the vehicle is likely to en…

This particular case might have been testable "in-situ":

[ABS News Reporter] Dan Noyes also spoke and texted with Walter Huang's brother, Will, today. He confirmed Walter was on the way to work at Apple when he died. He also makes a startling claim — that before the crash, Walter complained "seven-to-10 times the car would swivel toward that same exact barrier during auto-pilot. Walter took it into dealership addressing the issue, but they couldn't duplicate it there."

It is very believable that the car would swivel toward the same exact barrier on auto-pilot.

BTW - I'm running a nonprofit/public dataset project aimed at increasing safety of autonomous vehicles. If anyone here wants to contribute (with suggestions / pull requests / following it on twitter / etc) - you'd be most welcome. Its: https://www.safe-av.org/

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#328

Post by megablast »

buvanshak wrote: post #303
FireBeyond wrote: post #212

Earlier quoted context omitted.

Meanwhile, Tesla is busy putting out press releases saying "We believe the driver was inattentive, and the car was warning him to put his hands on the wheel." I am utterly, completely lacking in surprise that they didn't provide the relevant context, "... fifteen minutes prior." This just looks... really bad for Tesla. It's more important to them to protect their poor software than drivers.

If a company is this blatantly evil while still in a growing phase, I can only imagine what they would do once they are a established major player...

It is a car company. Their products directly kill over a million people every year. The injure 10x as many. Their pollution causes a similar number of deaths. They are responsible for so many horrible side effects, including dangerous city and suburbs to people taking up more and more space.

You think this one crash makes them evil?

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#329

Post by Robotbeat »

userbinator wrote: post #306

This just reconfirms my belief about Tesla's "autopilot" --- most of the time it behaves like an OK driver, but occasionally makes a fatal mistake if you don't pay attention and correct it. In other words, you have to be more attentive to drive safely with it than without, since a normal car (with suspension and tires in good condition, on a flat road surface) will not decide to change direction unless explicitly dir…

You can't make that kind of conclusion from a single, extremely-highly-publicized occurrence, even a fatal one. Given how often human drivers kill people, you need statistics to show that autopilot is worse.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#330

Post by vilhelm_s »

zo1 wrote: post #105
raldi wrote: post #76

Earlier quoted context omitted.

> The paint on the right edge of the gore marking, as seen in Google Maps, is worn Indeed; here's a close-up of the critical area. Note that the darker stripe on the bottom of the photo is NOT the crucial one; the one that the car was supposed to follow is the much more faded one above that that you can barely see: https://www.google.com/maps/@37.410912,-122.0757037,3a,75y,2... (Note that I'm not blaming the faded pa…

Wow. The start of that concrete lane-divider looks incredibly dangerous for a highway. It pretty-much just "starts", with only some sort of object that they call a "crash attenuator" in front of it. I would imagine its purpose is to dampen/dissipate kinetic energy and maybe deflect oncoming objects from hitting the concrete slab directly. I don't even see any sort of road-bumps to warn drivers of this dangerous obsta…

I had never heard of crash attenuators before, but they seem really impressive! This one was the "SCI smart cushion" model. According to the manual[1] it can reduce the acceleration from a collision at 100 km/h (63 mph) to less than 10G, so even a direct frontal collision at highway speed would be survivable[1]. That's pretty amazing.

[0] http://4fhmzg4ct6wezwll3wmp9kl9.wpengine.netdna-cdn.com/wp-c... [1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.212...

Post reply on HN