A Tragic Loss • Hacker News Board

Re: A Tragic Loss

#211

Post by johansch » Thu, Jun 30, 2016, 9:59 PM UTC

jacquesm wrote: post #94

This is why driving AI is 'all or nothing' for me. Assisted systems will lead to drivers paying less attention as the systems get better. The figures quoted by Tesla seem impressive but you have to assume the majority of the drivers is still paying attention all the time . As auto-pilots get better you'll see them paying attention less and then the accident rate will go up, not down for a while at least until the bug…

Volvo agrees with you:

http://jalopnik.com/volvo-engineer-calls-out-tesla-for-dange...

Re: A Tragic Loss

#212

Post by Retra » Thu, Jun 30, 2016, 9:59 PM UTC

return0 wrote: post #141

BookmarkSaver wrote: post #86

Earlier quoted context omitted.

>Tesla should not allow autopilots. They have nothing to gain by the negative publicity they will get from cases like these. This is literally the only way to develop such systems. And regardless of incidents like this, these systems will be developed sometime in the fairly near future and whichever company is on the leading edge of them will command a massive new market that will probably start growing incredibly ra…

They should at least let the drivers sign a contract that they 'll be putting themselves in danger.

Such a document exists. It's commonly called a driver's license.

Re: A Tragic Loss

#213

Post by nernst » Thu, Jun 30, 2016, 9:59 PM UTC

neurotech1 wrote: post #207

jacquesm wrote: post #94

This is why driving AI is 'all or nothing' for me. Assisted systems will lead to drivers paying less attention as the systems get better. The figures quoted by Tesla seem impressive but you have to assume the majority of the drivers is still paying attention all the time . As auto-pilots get better you'll see them paying attention less and then the accident rate will go up, not down for a while at least until the bug…

In the aviation community, there is the major concerns over pilots becoming over-reliant cockpit automation instead of flying the jet. Asiana 214 [0] is a classic example of crashing a perfectly good airliner into a seawall on landing. In the Boeing 777, one example of the (auto)pilot interface showing safety critical information is the stall speed indication on the cockpit display [1], warning the pilot if they are…

My understanding of the Asiana crash was that the autopilot would have landed the plane fine, and that it was the humans turning it off that caused the problem.

Your point is still valid, but perhaps we approach a time when over-reliance is better than all but the best human pilots (Sully, perhaps).

Re: A Tragic Loss

#214

Post by michalnaka » Thu, Jun 30, 2016, 10:01 PM UTC

niftich wrote: post #58

Other premium automakers have automatic braking and pre-collision systems that take effect regardless of whether the (enhanced) cruise control is engaged. We have seen from other incidents that Tesla's autobraking will not engage when autopilot is disabled (potentially by tapping the brakes), which is contrary to widespread user expectation -- disclaimers notwithstanding. However, in this particular case, it was the…

Which manufacturers use eye-height radar-based systems?

Re: A Tragic Loss

#215

Post by neurotech1 » Thu, Jun 30, 2016, 10:02 PM UTC

ghaff wrote: post #194

jacquesm wrote: post #94

This is why driving AI is 'all or nothing' for me. Assisted systems will lead to drivers paying less attention as the systems get better. The figures quoted by Tesla seem impressive but you have to assume the majority of the drivers is still paying attention all the time . As auto-pilots get better you'll see them paying attention less and then the accident rate will go up, not down for a while at least until the bug…

It's pretty well established that humans (e.g. [1]) that humans have a lot of trouble paying when they don't need to be actively engaged most of the time and also have trouble taking back control. In a consumer driving context, I have zero doubt that, as systems like these develop, people will start watching videos and reading absent draconian monitoring systems to ensure they keep their eyes on the road. I'm not sur…

Its worth pointing out that Prof. Missy Cummings, who authored the paper, is a former F/A-18 pilot who specializes in human-machine interaction research.

One option is the Tesla autopilot should have an indication when it approaches "low confidence" areas without disengaging, so the driver is not startled if they have to take back manual control.

Re: A Tragic Loss

#216

Post by Animats » Thu, Jun 30, 2016, 10:02 PM UTC

"Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied."

Now that's blaming the user for a design flaw. There is no excuse for failing to detect the side of a semitrailer. Tesla has a radar. Did it fail, was its data misprocessed, or is the field of view badly chosen? The single radar sensor is mounted low on the front of the vehicle, and if it lacks sufficient vertical range, might be looking under a semitrailer. That's no excuse; semitrailers are not exactly an uncommon sight on roads.

I used an Eaton VORAD radar in the 2005 Grand Challenge, and it would have seen this. That's a radar from the 1990s.

I want to see the NTSB report. The NTSB hasn't posted anything yet.

Re: A Tragic Loss

#217

Post by ars » Thu, Jun 30, 2016, 10:02 PM UTC

angusb wrote: post #123

BinaryIdiot wrote: post #21

I mean I understand Tesla has to make a statement here and I understand they want to ensure everyone that it's not really their fault but to title a post "A Tragic Loss" and then spend the majority of the post discussing all of your car's safety features and how it wasn't your fault just seems tone deaf and distasteful to me. Maybe they had to do it for legal reasons I don't know (I'm certainly not a lawyer) and I'd…

It's not just about Tesla though. They have to defend autopilot not only to protect the brand but to protect the public's perception of autonomous vehicles in general. Self driving tech is poised to save many many lives. So from a utilitarian perspective, it's probably justified to take extraordinary measures to make sure reactionary media and public whim doesn't kill it off, however uncomfortable that might seem in…

> Self driving tech is poised to save many many lives.

The computer would have to be 99.99999% reliable to do that.

The accident rate is around 74 per 100 million miles (and fatalities is 1.13).

It's unclear exactly how to turn that into a percentage, but no matter how you do it it's quite high.

Say an accident takes 5 minutes, and people drive 30 miles/hour. Then that works out to 99.999% for humans. If you use the numbers for fatalities then it's 99.99999%.

I.e. 99.99999% of the time, as whole across all [US] humans, people drive in a way that does not cause a fatality.

That's the bar computers have to cross in order to save any lives at all.

Re: A Tragic Loss

#218

Post by stcredzero » Thu, Jun 30, 2016, 10:03 PM UTC

SapphireSun wrote: post #91

georgecmu wrote: post #25

Earlier quoted context omitted.

Autopilot engaged when a tractor trailer drove across the highway perpendicular to the Model S. Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied. How is this not an "Autopilot did something very very wrong" story? Tesla autopilot drove the vehicle full-speed into a tractor-trailer, which wasn't registered by its sensor systems. It'…

I guess the question is, was the driver able to see it? I can see how maybe the contrast was so low that maybe it was impossible to see anything except for the wheels which you might not notice. And if the part of the tractor trailer in the path is elevated... sensors sweeping forward see just road. I guess I'd like to see an NHTSB simulation of the situation in manual driving to see how humans do unassisted.

And if the part of the tractor trailer in the path is elevated... sensors sweeping forward see just road.

It could be even worse. If there was a "road surface mirage" then it's possible that both the truck and the road underneath it were not visible at all. Since the forward radar doesn't have the range or coverage to see a truck at a distance, the only sensor that could have detected the problem was the camera. Because of the phenomena I've mentioned above and the parent post has mentioned, we know this can't possibly work.

Even worse, the circumstances in which the car's sensor could fail are precisely the ones in which the Mark 1 Eyeball would also fail.

Re: A Tragic Loss

#219

Post by ars » Thu, Jun 30, 2016, 10:04 PM UTC

kafkaesq wrote: post #172

This is the first known fatality in just over 130 million miles where Autopilot was activated. Among all vehicles in the US, there is a fatality every 94 million miles. Given that Autopilot is generally activated under safer, more routine driving scenarios -- decent weather, regular suburban traffic and so on; for which we can naturally expect significantly lower fatality rates -- it doesn't sound like it's off to a…

Exactly. And on top of that, how much of the time was it the human who actually prevented the fatality, and not autopilot?

I would like to see how many accidents would have happened under Autopilot, but were only prevented because of the human.

Re: A Tragic Loss

#220

Post by kamaal » Thu, Jun 30, 2016, 10:05 PM UTC

krschultz wrote: post #85

mikeash wrote: post #19

Earlier quoted context omitted.

From the information given, we don't know if the truck pulled out right in front of the car giving the driver no time to react, or if the truck pulled out a minute beforehand and the driver was just oblivious. In the absence of any information either way, it's nasty to just assume the driver was inattentive.

I don't think it's nasty, I think it's realistic. I also think that it aligns with the research that shows humans take a significant amount of time to adjust to taking back over control from autopilot. This is one of the primary design considerations behind the Google self driving car having absolutely no steering wheel or brake. The illusion of humans being able to be the fallback when they're not physically capable…

>>This is one of the primary design considerations behind the Google self driving car having absolutely no steering wheel or brake.

Didn't know about this. If they are truly testing their software in a car without break and steering wheel I have a whole new level of respect for them.

When you start to design a self-driving car the question of a human fallback shouldn't even exist. The car either drives itself, or humans do. Mixing responsibilities in situations of life and death doesn't end well.