NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash • Hacker News Board

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#141

Post by drag0nballz » Thu, Jun 07, 2018, 6:00 PM UTC

TheLoneAdmin wrote: post #83

Animats wrote: post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

Makes me wonder about the future of self-driving cars. I would think that the first thing that should be programmed in is to not run into objects that are not moving. IMO, maybe the roads need to be certified for self-driving. Dangerous areas would be properly painted to prevent recognition errors. Every self-driving system would need to query some database to see if a road is certified. If not, the self-driving syst…

This is a Tesla issue. Waymo uses Lidar and most likely won't run into walls.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#142

Post by jakelarkin » Thu, Jun 07, 2018, 6:01 PM UTC

Animats wrote: post #44

jakelarkin wrote: post #14

self driving systems cant well reason about untrained scenarios or the intent of other humans on the road. I think the people have grossly underestimated how driving in an uncontrolled environment is really a general AI problem, which we're not even close to solving.

This wasn't a hard case that needed advanced AI. This was a clearly visible stationary obstacle on a good freeway on a clear California day.

deciding how/when to make panic stop or evasive maneuvers without routinely becoming a hazard requires advanced AI

determining the path of travel through lanes markings that suddenly fade out while moving in 70mph on a busy freeway requires advanced AI

its pretty clear the self-driving gods are not giving these cars a grounds up understanding of physics and self-preservation. They best thing they come up with is programmed to drive X mph and to respond probabilistically to a large but finite set of training input scenarios.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#143

Post by d0lph » Thu, Jun 07, 2018, 6:01 PM UTC

jumelles wrote: post #92

d0lph wrote: post #65

Earlier quoted context omitted.

Or perhaps us fingers-in-ears Musk fans don't expect the AP to be perfect. Especially considering how new the fields are. Autopilot doesn't need to be perfect, just better than humans.

So because the field is new, it's okay that a human being is dead?

If human beings dying weren't at least somewhat ok, we wouldn't drive cars.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#144

Post by [deleted] » Thu, Jun 07, 2018, 6:03 PM UTC

slg wrote: post #137

txcwpalpha wrote: post #117

Earlier quoted context omitted.

And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal flying at 70mph with humans inside…

This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and…

[deleted]

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#145

Post by Animats » Thu, Jun 07, 2018, 6:03 PM UTC

baconmania wrote: post #61

Animats wrote: post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

What's your source for the claim that Tesla's system "doesn't detect stationary objects"? From the reference frame of a moving Tesla, both globally stationary and globally moving objects will appear to be in motion.

Teslas on autopilot have collided with many stationary objects that were partially blocking a lane. Known incidents include a street sweeper at the left edge of a highway (China, fatal, video available), a construction barrier in the US (video), a fire truck in the SF bay area (press coverage), a stalled car in Germany (video), a crossing semitrailer (NTSB investigation), and last month, a fire company truck in Utah.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#146

Post by txcwpalpha » Thu, Jun 07, 2018, 6:03 PM UTC

slg wrote: post #137

txcwpalpha wrote: post #117

Earlier quoted context omitted.

And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal flying at 70mph with humans inside…

This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and…

>This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software.

Cars have been dependent on software for a long time (literally decades). This isn't something new. Even combustion engine cars have had software inside of them that controls the operation of the engine, and this software is vigorously tested for safety issues (because most car manufacturers understand a fault with such software could result in someone's death). Tesla seems to be the only major car manufacturer that has a problem with this.

>So what is the right way to handle these updates?

The way that other vehicle manufacturers (car, airplane, etc) have been doing it for decades is a pretty good way.

>You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer.

There is no evidence that said OTA update made Tesla cars any safer. There is evidence that similar OTA updates have made Tesla cars more unsafe.

The brake OTA that you mentioned has actually potentially done more harm than good. Tesla owners have been reporting that the same update made unexpected changes to the way their cars handle/accelerate in addition to the change in braking distance. These were forced, unpredictable changes that were introduced without warning. When you're driving a 2 ton vehicle at 70mph, being able to know exactly how your car will react in all situations, including how fast it accelerates, how well it handles, how fast it brakes, and how the autopilot will act is crucial to maintaining safety. Tesla messing with those parameters without warning is a detriment to safety, not an advantage.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#147

Post by brigade » Thu, Jun 07, 2018, 6:04 PM UTC

rhacker wrote: post #49

martin_bech wrote: post #32

Earlier quoted context omitted.

A counter argument that HUMAN DRIVERS are unsafe an fatal (let alone missing common sense and alertness). I 100% get your point, but my counterpoint, is that this is what we are going to see, in the future. Autonomous systems that by and large are way safer, than human drivers, but when they fail, the might do so, in circumstances where are human would not. Humans are good at some things, computers/ai/sensor suites a…

I get that in 5-10 years we MAY have SDVs that are superior than human drivers. But I think a lot of people have a mental model of SDVs _right now_ being superior to human drivers. I can't remember the crash stats, but I think the only way to know for sure is to calculate the number of miles a single human generally drives on average before they are involved in a collision or crash, etc... Now I don't have the number…

Tesla might not be advertising L5, but they're certainly claiming that Autopilot is safer than human drivers right now.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#148

Post by crote » Thu, Jun 07, 2018, 6:07 PM UTC

panopticon wrote: post #127

zo1 wrote: post #105

Earlier quoted context omitted.

Wow. The start of that concrete lane-divider looks incredibly dangerous for a highway. It pretty-much just "starts", with only some sort of object that they call a "crash attenuator" in front of it. I would imagine its purpose is to dampen/dissipate kinetic energy and maybe deflect oncoming objects from hitting the concrete slab directly. I don't even see any sort of road-bumps to warn drivers of this dangerous obsta…

Out of curiosity, are you not in the U.S.? These types of lane dividers are incredibly common throughout the country. Note that I'm not defending the safety of them; I'm just surprised to see someone call them out as a hazard as they're such a common sight.

Dutch citizen here, I have never seen something like this.

To compare, here's a similar situation in The Netherlands:

https://www.google.nl/maps/@52.0928662,5.1734685,3a,75y,281....

Points of interest:

- big solid white arrow, line-fade is nearly impossible

- white-green sign indicating that the road is splitting

- big shoulder in the continuation of the division

- loads of grass instead of immediately using a metal barrier

- gently rising metal barrier, so driving straight into it will result in something like this: http://cdn.brandweersassenheim.nl/large/269dezilkvangrail104...

Same goal, but a completely different approach.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#149

Post by EthanHeilman » Thu, Jun 07, 2018, 6:08 PM UTC

TheLoneAdmin wrote: post #83

Animats wrote: post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

Makes me wonder about the future of self-driving cars. I would think that the first thing that should be programmed in is to not run into objects that are not moving. IMO, maybe the roads need to be certified for self-driving. Dangerous areas would be properly painted to prevent recognition errors. Every self-driving system would need to query some database to see if a road is certified. If not, the self-driving syst…

>Makes me wonder about the future of self-driving cars. I would think that the first thing that should be programmed in is to not run into objects that are not moving.

My understanding is that Telsa's autopilot is pretty different from other more mature self-driving car projects. I wouldn't read too much into it.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#150

Post by zaroth » Thu, Jun 07, 2018, 6:09 PM UTC

metalliqaz wrote: post #139

Animats wrote: post #42

NTSB: • At 8 seconds prior to the crash, the Tesla was following a lead vehicle and was traveling about 65 mph. • At 7 seconds prior to the crash, the Tesla began a left steering movement while following a lead vehicle. • At 4 seconds prior to the crash, the Tesla was no longer following a lead vehicle. • At 3 seconds prior to the crash and up to the time of impact with the crash attenuator, the Tesla’s speed increas…

But why would it speed up beyond the speed limit?

Because the driver told it to.