Live data from Hacker News

Video suggests huge problems with Uber’s driverless car program

arstechnica.com

131–140 of 244 posts

Re: Video suggests huge problems with Uber’s driverless car program

#131

Why don’t we require software engineers who work on self driving car software to go through licensing and certification. And then, if their code results in a death, they are liable and can have their license completely revoked, and they would be unable to work on self driving cars again.

Uber will relocate the engineering team to a jurisdiction without those regulations. Just like it relocated its testing to get a more "business friendly regulatory environment".

That's trivially fixed by requiring the engineering to be certified in the jurisdiction in which its output operates.

For a mostly topical example of such rules: https://www.dmv.ca.gov/portal/dmv/detail/vr/checklists/outof...

Re: Video suggests huge problems with Uber’s driverless car program

#132
post #124
post #99

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1]. > To get to that better software faster we should deploy the first 1000 cars asap. Is that admitting that they do not have t…

Is it better for self driving cars to have a flawless record & low adoption, or to have a 100x improvement over human drivers & broad adoption? Creating the perfect self-driving car, with redundant systems, safety everything & so on will certainly help its safety records. But it will also drive up the cost. And put it out of reach for a lot of people. If the goal is to save lives, the bar self-driving cars should be…

From the article:

> But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States.

I don't think there's enough evidence to say that self-driving cars are as safe as humans.

Re: Video suggests huge problems with Uber’s driverless car program

#134
post #17
post #14

Earlier quoted context omitted.

My guess is that it's because the field is so new that there aren't really any experts that can define what are reasonable rules for said licensing and certifications

They exist. They're required for some roles: https://insight.ieeeusa.org/articles/professional-licensure-...

The software engineering P.E. exam is being discontinued after the April 2019 administration, due to low turnout: only 81 candidates since 2013.

https://ncees.org/ncees-discontinuing-pe-software-engineerin...

Re: Video suggests huge problems with Uber’s driverless car program

#135
post #76

"Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States." Wow there goes that "safer than human drivers" argument.

With one data point, you can't extrapolate much. This is misuse of statistics. Consider if there was a new lottery and you weren't sure what the odds of winning were. You play it three weeks in a row and the third time you win a million dollars. Conveniently, no one else tries the new lottery yet. Does it follow then that the odds of winning a million dollars are 1 in 3? Or should you play it a few more times before…

It is not a misuse of statistics for "one data point" to significantly shift our beliefs. Let's do the math.

Bayesian approach: To make the math really simple, let's assume a discrete prior on Uber's death rate. Say 33% that Uber's cars are much safer than humans (0.1 deaths per 100M miles), 33% that they are equally safe (1 death per 100M miles), and 33% that they are much more dangerous (10 deaths per 100M miles). After observing one death at 3 million miles, your posterior is should update to {safer: 1%, equal: 11%, more dangerous: 88%). This is a substantial shift in confidence.

Math: http://www.wolframalpha.com/input/?i=(1-10%2F100)%5E2*(10%2F...)

Frequentist approach: Let the null hypothesis be that Uber's self-driving cars have the same death rate as humans - 1 death per 100 million miles. The odds of Uber killing someone within 3 million miles is about 3%. Therefore, we can reject the null hypothesis with a p value of 0.03. One positive "data point" is statistically significant.

Statistically, one death after 3 million miles is not proof that Uber's death rate is higher than 1 in 100 million miles. But it is statistically significant, in both a frequentist and Bayesian framework. You have to get really, really unlucky to have a death at 3 million miles if your death rate is 1 per 100 million miles.

Bottom line: This collision isn't proof, but it's strong evidence. (To go along with all the evidence from crash rates, disengagement rates, engineers working at these companies, and the video of the crash itself.)

Re: Video suggests huge problems with Uber’s driverless car program

#136
post #124
post #99

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1]. > To get to that better software faster we should deploy the first 1000 cars asap. Is that admitting that they do not have t…

Is it better for self driving cars to have a flawless record & low adoption, or to have a 100x improvement over human drivers & broad adoption? Creating the perfect self-driving car, with redundant systems, safety everything & so on will certainly help its safety records. But it will also drive up the cost. And put it out of reach for a lot of people. If the goal is to save lives, the bar self-driving cars should be…

[deleted]

Re: Video suggests huge problems with Uber’s driverless car program

#137
post #114
post #102

Earlier quoted context omitted.

If you're looking for the rate at which something happens, for the purpose of predicting events in the future, you need to see the event occur many times before you can fairly estimate it's rate. That's what my example means. I'm not defending Uber here- I'm defending statistics! With more data, we may discover that Uber cars are 100X worse, not just 25X. Or we may discover they're better. But we don't have the stati…

Your knowledge of statistics could use some enhancement. The response referring to poisson processes is a better view of things.

> Your knowledge of statistics could use some enhancement.

In this we agree.

Re: Video suggests huge problems with Uber’s driverless car program

#138
post #48

Earlier quoted context omitted.

- Expecting engineers to always write perfect code is insane. Mistakes happen. - If bad code makes it into production, that is a systemic failure not an individual one (Why didn't the bug get caught in code review, QA, etc.) - No one is going to want to work on a project where a single failure can taint their career. - What if I use a 3rd party lib and that is where the bug is. Who is at fault then? What if the code…

There is ISO26262 for automotive about safety. Yes, there are reviews, QA, and all of that. So, yes, there is no single person responsible (exceptions apply). But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it. That is the reason, why certifications are for. The same rules apply for medical and other areas.

> But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it.

Pretty much everything in development relies on the work of other people. I used 3rd party lib just as an example, but what if it's in the framework or even the language that an engineer uses, who would be at fault then? You can't expect every developer to have gone through the entire source code for whatever language they are writing in.

Sciences build on each other and and after a certain period of time you have to take things for granted in order to keep moving forward.

> The same rules apply for medical and other areas.

No, they don't. Doctors kill patients all the time and they aren't banned from medicine for it. There is an investigation, they make sure it wasn't intentional and there wasn't any gross negligence and that this isn't a repeating pattern, if none of those are the case they see what they can learn from it and move forward in hopes that what they learn can help other doctors.

Re: Video suggests huge problems with Uber’s driverless car program

#139

Earlier quoted context omitted.

Your concept that there are no accidents, only oversights, is not correct. Say I’m crossing the street and I don’t see your car coming, because I look the wrong way or I’m distracted, or my view is blocked by the sun. You may not be able to avoid hitting me. Additionally, if we train self-driving cars to always give way to pedestrians who even look like they might cross the street, they’re going to have a heck of a t…

> Your concept that there are no accidents, only oversights, is not correct. Say I’m crossing the street and I don’t see your car coming, because I look the wrong way or I’m distracted, or my view is blocked by the sun. You may not be able to avoid hitting me Solvable by reducing speed sufficiently. It's reasonable to expect the car to avoid a man travelling at 10kph from any off-road blind-spot, and a car travelling…

It’s not a blind spot I’m worried about. People frequently walk at full speed up to the edge of the sidewalk, then stop just before they would walk into traffic. Should an autonomous car assume that any pedestrian walking towards an intersection is going to continue into the roadway, even if they don’t have the right of way? That’s not what a human driver does.

Likewise, I can be standing still with my toes on the curb, and then lunge into the street. Should a self-driving car assume that every pedestrian standing at a crosswalk could walk into traffic at any moment, and slow down accordingly? Again, that’s not what human drivers do.

There are a number of surface streets near my house with speed limits of 45 mph, and crosswalks every 1/8 mile or so. Requiring cars (autonomous or not) to avoid any possible pedestrian incident at every such intersection would be a disaster for traffic throughout and a huge step backwards.

Re: Video suggests huge problems with Uber’s driverless car program

#140
post #70
post #5

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski to Alphabet CEO Larry Page in January 2016. Looks like Uber has attracted Levandowski due to his cultural fit.

Hmm, but wouldn’t his priorities be correct in the context of this crash? There hasn’t been any suggestion (so far) that the crash occurred because some hardware component stopped working; rather, it seems like the software failed to identify the pedestrian in time. So better software seems precisely what was needed. Though I can imagine that better sensors might also have helped…

The issue is not that he wanted better software, it's that he appeared willing to compromise safety to get it faster in order to beat his competitors to market, as is clear from the remainder of that quote:

"To get to that better software faster we should deploy the first 1000 cars asap. I don't understand why we are not doing that. Part of our team seems to be afraid to ship."

And from another email:

"the team is not moving fast enough due to a combination of risk aversion and lack of urgency"

Post reply on HN