Video suggests huge problems with Uber’s driverless car program • Hacker News Board

Re: Video suggests huge problems with Uber’s driverless car program

#171

Post by Lewton » Fri, Mar 23, 2018, 6:32 AM UTC

Fricken wrote: post #118

slavik81 wrote: post #109

Earlier quoted context omitted.

One accident is clearly not one data point. If Uber had driven a billion miles with 0 accidents, we would safely conclude they were safer than human drivers with "0 data points". Assuming that accidents are independent, we can model this as a Poisson point process. If the accident rate is 1 per 100M miles and Uber has driven 3M miles, the probability of there being zero accidents in that time is P{n=0} = ((λt)^n / n!…

Gil Pratt, who heads up Toyota's autonomous development initiative mentioned that we would need to drive 8.5 billion autonomous miles to be able to declare with 99% statistical certainty that autonomous vehicles are safer that human driven ones. Of course, as we are witnessing, great pains will be taken with every preventable injurious or fatal collision to ensure that sort of failure never happens again, so by the t…

Other big players have taken great pains to ensure that it wouldn’t happen in the first place

Uber should be banned from doing AI research on public roads

Re: Video suggests huge problems with Uber’s driverless car program

#172

Post by akkat » Fri, Mar 23, 2018, 6:37 AM UTC

JohnJamesRambo wrote: post #62

"Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States." Wow there goes that "safer than human drivers" argument.

I don't know how to say it kindly but there is a difference between "what" type of people the car killed. If the fatality was another rule abiding driver on the road or a pedestrian crossing at a crosswalk that would be really bad. However if it was someone not following the safety rules by j walking, then that person accepted upon themselves a higher probability of being in an accident. When making laws, for the most part, they are for the benefit of law abiding people.

Re: Video suggests huge problems with Uber’s driverless car program

#173

Post by culturestate » Fri, Mar 23, 2018, 6:51 AM UTC

AlexandrB wrote: post #153

jon_richards wrote: post #141

Earlier quoted context omitted.

To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. Yes, you lose some life in the interim, but you accelerate the creation and adoption of something that saves much more life in the long run. But that is human experimentation, something we as a culture generally agree is abhorrent.

> To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. This seems unreasonably optimistic. First, this particular crash is an egregious counter-example. The car doesn't even seem to slow down when it first sees the pedestrian's foot. Nor does it try to swerve. This is basic stuff for a human driver, never mind more complex avoidance and risk mitigation…

> First, this particular crash is an egregious counter-example. The car doesn't even seem to slow down when it first sees the pedestrian's foot. Nor does it try to swerve. This is basic stuff for a human driver, never mind more complex avoidance and risk mitigation a human driver can perform.

I'm not sure which side of this I fall on just yet, but something strikes me here: you're assuming a human driver with no impairment (e.g. eyesight or fatigue) paying complete attention to what they're doing. We know that this isn's always the case (and could even be a minority!), so this doesn't seem to be a great argument.

Re: Video suggests huge problems with Uber’s driverless car program

#174

Post by blt » Fri, Mar 23, 2018, 6:51 AM UTC

jon_richards wrote: post #141

discodave wrote: post #132

Earlier quoted context omitted.

From the article: > But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed…

To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. Yes, you lose some life in the interim, but you accelerate the creation and adoption of something that saves much more life in the long run. But that is human experimentation, something we as a culture generally agree is abhorrent.

No. Machine learning is not magic. Machine learning has nothing to say when the objective function is not well understood.

Re: Video suggests huge problems with Uber’s driverless car program

#175

Post by toast0 » Fri, Mar 23, 2018, 6:54 AM UTC

ddeck wrote: post #140

comex wrote: post #70

Earlier quoted context omitted.

Hmm, but wouldn’t his priorities be correct in the context of this crash? There hasn’t been any suggestion (so far) that the crash occurred because some hardware component stopped working; rather, it seems like the software failed to identify the pedestrian in time. So better software seems precisely what was needed. Though I can imagine that better sensors might also have helped…

The issue is not that he wanted better software, it's that he appeared willing to compromise safety to get it faster in order to beat his competitors to market, as is clear from the remainder of that quote: "To get to that better software faster we should deploy the first 1000 cars asap. I don't understand why we are not doing that. Part of our team seems to be afraid to ship." And from another email: "the team is no…

The rest of the quote is much more powerful. It's pretty irresponsible to ship 1000 self driving cars onto public roads at this point. (Regardless of who is shipping them)

On the other hand, redundant steering and braking seem like probable overengineering. Brakes are already somewhat redudnant (dual section master cylinders were common in the 70s and are almost certainly in any modern vehicle), and better software could periodically verify they're working and if not, coast to a stop. Steering failure could be handled by engaging the brakes. Simultaneous failure is likely rare and catastrophic anyway -- losing a wheel and having the brakes pressure go with it can happen, and when it does, you put on your blinkers and hope you come to rest in a safe manner.

Re: Video suggests huge problems with Uber’s driverless car program

#176

Post by Glawen » Fri, Mar 23, 2018, 6:56 AM UTC

superfrank wrote: post #138

PinguTS wrote: post #48

Earlier quoted context omitted.

There is ISO26262 for automotive about safety. Yes, there are reviews, QA, and all of that. So, yes, there is no single person responsible (exceptions apply). But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it. That is the reason, why certifications are for. The same rules apply for medical and other areas.

> But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it. Pretty much everything in development relies on the work of other people. I used 3rd party lib just as an example, but what if it's in the framework or even the language that an engineer uses, who would be at fault then? You can't expect every developer to have gone through the entire source code for whatever languag…

That's why in iso 26262 you can only use certified compiler and tools. Rust compiler is not certified for example. Sure you can use 3rd party libs, but they must be certified.

I agree dith parent: never use copy paste from internet in safety critical SW, anyway it most probably isn't designed for your use cade. Personally I always have been disapointed by copy pasting stuff, it was always buggy somehow. In the end I always reimplemented it from scratch by reading the theory

Re: Video suggests huge problems with Uber’s driverless car program

#177

Post by joejerryronnie » Fri, Mar 23, 2018, 7:21 AM UTC

akkat wrote: post #172

JohnJamesRambo wrote: post #62

"Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States." Wow there goes that "safer than human drivers" argument.

I don't know how to say it kindly but there is a difference between "what" type of people the car killed. If the fatality was another rule abiding driver on the road or a pedestrian crossing at a crosswalk that would be really bad. However if it was someone not following the safety rules by j walking, then that person accepted upon themselves a higher probability of being in an accident. When making laws, for the mos…

This is a ridiculous statement.

Re: Video suggests huge problems with Uber’s driverless car program

#178

Post by AlotOfReading » Fri, Mar 23, 2018, 7:22 AM UTC

KKKKkkkk1 wrote: post #154

buildbot wrote: post #91

I don't understand how there isn't a non-ML based piece of code that looks at moving radar and lidar returns and preforms an emergency brake, light flash, horn, or dodge if that vector would intersect with any confidence. Even slowing down to 20MPH can turn a fatal accident into an injury.

What if it has nothing to do with ML? You see a point cloud that's moving toward your lane at a speed estimate of say 2 mph. If that's below the sensor noise threshold, you might classify the cloud as a stationary object on the other lane (say a stranded car). In that case, by the time you realize that this stationary object has somehow moved itself into your own lane, it is already too late.

You slow down anyways, because it will almost always improve the situation and reduce your liability. Secondly, if the vision is so bad that it can't identify objects in its lane or predict vectors, it's not ready for the real world.

Re: Video suggests huge problems with Uber’s driverless car program

#179

Post by phlo » Fri, Mar 23, 2018, 7:23 AM UTC

pdkl95 wrote: post #99

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1]. > To get to that better software faster we should deploy the first 1000 cars asap. Is that admitting that they do not have t…

> willful contempt for safety

To play Kalanick's adversary, he might be arguing for more real-world data collection. Tesla famously equipped most of their cars with more sensors than were required at the time of delivery, using the data to drive development of the Autopilot function that was later added to the cars.

Re: Video suggests huge problems with Uber’s driverless car program

#180

Post by cmac2992 » Fri, Mar 23, 2018, 7:27 AM UTC

troupe wrote: post #94

mabbo wrote: post #76

Earlier quoted context omitted.

With one data point, you can't extrapolate much. This is misuse of statistics. Consider if there was a new lottery and you weren't sure what the odds of winning were. You play it three weeks in a row and the third time you win a million dollars. Conveniently, no one else tries the new lottery yet. Does it follow then that the odds of winning a million dollars are 1 in 3? Or should you play it a few more times before…

If you do want to extrapolate from their data, it would be worth looking at the total number of accidents--not just fatalities. Insurance companies say people file an accident claim every 18 years on average. If the average miles someone drives each year is 12,000 miles, this means they get in an accident every 216,000 miles on average. If Uber drove 3 million miles, we should expect them to have been involved in abo…

You'd also need to account for disengagements. Some portion of all disengagements were likely to avoid accidents.