Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video suggests huge problems with Uber’s driverless car program

arstechnica.com

Post reply on HN
Print view View original

151–160 of 244 posts

Re: Video suggests huge problems with Uber’s driverless car program

#151

Post by JKCalhoun »

iClaudiusX wrote: post #123
pdkl95 wrote: post #99

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1]. > To get to that better software faster we should deploy the first 1000 cars asap. Is that admitting that they do not have t…

I get the feeling based on comments here that there is a severe lack of ethical and critical thinking among engineers and developers. I recognize that this is only a vocal minority but the constant mantra of "move fast and break things", where getting rich at any cost is seen as a virtue, has made me extremely disillusioned with this brand of startup culture. Doubly so when people are trading stock tips on how to pro…

I'm not sure the "self-driving car industry" can survive these possible (inevitable?) future scenarios:

• Self-driving car kills child

• Hackers send self-driving car on wild ride

• Empty self-driving semi found in warehouse lot, GPS pirates make off with entire cargo

Re: Video suggests huge problems with Uber’s driverless car program

#152

Post by fourthark »

jon_richards wrote: post #141
discodave wrote: post #132

Earlier quoted context omitted.

From the article: > But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed…

To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. Yes, you lose some life in the interim, but you accelerate the creation and adoption of something that saves much more life in the long run. But that is human experimentation, something we as a culture generally agree is abhorrent.

This is assuming that the learning actually happens.

Re: Video suggests huge problems with Uber’s driverless car program

#153

Post by AlexandrB »

jon_richards wrote: post #141
discodave wrote: post #132

Earlier quoted context omitted.

From the article: > But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed…

To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. Yes, you lose some life in the interim, but you accelerate the creation and adoption of something that saves much more life in the long run. But that is human experimentation, something we as a culture generally agree is abhorrent.

> To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans.

This seems unreasonably optimistic.

First, this particular crash is an egregious counter-example. The car doesn't even seem to slow down when it first sees the pedestrian's foot. Nor does it try to swerve. This is basic stuff for a human driver, never mind more complex avoidance and risk mitigation a human driver can perform.

Second, we've had years of training various AI content curation algorithms on social networks, videos, blogs, etc - and the most advanced AI and search company in the world still can't keep adult-oriented conspiracy videos off of Youtube Kids. And while you might counter that content is a human problem, driving is too! Dangerous driving situations happen at the periphery of traffic rules, where someone is doing something the drivers around him don't expect. I've seen people run solid red lights, drive the wrong way down a one-way street, pedestrians start crossing when their light turns red, etc. Short of creating special roads for autonomous cars only - how does an autonomous car deal with all of this successfully?

Re: Video suggests huge problems with Uber’s driverless car program

#154

Post by KKKKkkkk1 »

buildbot wrote: post #91

I don't understand how there isn't a non-ML based piece of code that looks at moving radar and lidar returns and preforms an emergency brake, light flash, horn, or dodge if that vector would intersect with any confidence. Even slowing down to 20MPH can turn a fatal accident into an injury.

What if it has nothing to do with ML? You see a point cloud that's moving toward your lane at a speed estimate of say 2 mph. If that's below the sensor noise threshold, you might classify the cloud as a stationary object on the other lane (say a stranded car). In that case, by the time you realize that this stationary object has somehow moved itself into your own lane, it is already too late.

Re: Video suggests huge problems with Uber’s driverless car program

#155

Post by Fricken »

jamesgeck0 wrote: post #146
Fricken wrote: post #118

Earlier quoted context omitted.

Gil Pratt, who heads up Toyota's autonomous development initiative mentioned that we would need to drive 8.5 billion autonomous miles to be able to declare with 99% statistical certainty that autonomous vehicles are safer that human driven ones. Of course, as we are witnessing, great pains will be taken with every preventable injurious or fatal collision to ensure that sort of failure never happens again, so by the t…

Maybe her death wasn't in vain, but it was definitely avoidable. If Uber rushes out half-baked driverless cars, fallout from the incidents they're responsible for will cause serious delays to widespread deployment. Trading lives to save on R&D time would violate professional codes of ethics in literally any other industry.

It wouldn't violate professional codes of ethics in a war, and we're taking wartime casualty numbers on our roads everyday.

This is the real trolly problem when it comes to the ethics of developing self driving cars.

Re: Video suggests huge problems with Uber’s driverless car program

#156

Post by henrikeh »

URSpider94 wrote: post #142
femto wrote: post #126

Earlier quoted context omitted.

I think your first paragraph is written from the perspective of a human driver. The machine can predict based on physics. If anything a person who does not see a car will be more predictable, not less, as they will tend to be moving uniformly and not taking evasive action. Taking into account optimal (computer driven) car stopping distances, the maximum acceleration of a person and typical pedestrian densities, I don…

My point is not that children would jump in front of cars. It’s that they would run toward the curb with a heading and velocity that would dictate a collision, then stop just before entering the street. They would do it for the main reason that children do things - because it would be hilarious to watch self-driving cars leave a patch on the road and jerk their passengers back with seatbelt pretensioners. If you don’…

In most countries obstructing the flow of traffic, no matter the method, is illegal.

And what alternative do you suggest? Not braking?

Re: Video suggests huge problems with Uber’s driverless car program

#157

Post by ggg9990 »

mabbo wrote: post #97
soyiuz wrote: post #81

Earlier quoted context omitted.

They did not sample three times and win once. They sampled three million times and won once. Hardly "one data point."

The point remains: it isn't fair to decide what the probability is with only one data point on one side- especially for rare events. Would it have been fair if Uber last week were to declare that they have a 0% probability of pedestrian deaths, since they'd never had one yet? The goal of these statistics is to predict future outcomes. But with such a small data sample, you cannot fairly predict the future- just as in…

What if Uber’s first self-driving car killed a cyclist in its very first mile of operation? Would you find it equally hard to draw conclusions?

Re: Video suggests huge problems with Uber’s driverless car program

#158

Post by henrikeh »

colatkinson wrote: post #122
PinguTS wrote: post #48

Earlier quoted context omitted.

There is ISO26262 for automotive about safety. Yes, there are reviews, QA, and all of that. So, yes, there is no single person responsible (exceptions apply). But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it. That is the reason, why certifications are for. The same rules apply for medical and other areas.

> But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it. Wait, what? That goes against one of the core benefits of open source software--that having many eyes on a problem decreases the risk of bugs. I'm willing to bet that if Uber had to implement their own machine learning/vision libraries from the ground up, there would be significantly more issues.

Is there any evidence that this is the case? That simply putting more eyes on a system will reveal its problems?

Certification etc is about process. Open source code can be used in a safety critical product, but it must audited and confirmed against the system requirements.

Re: Video suggests huge problems with Uber’s driverless car program

#159

Post by bobthepanda »

roenxi wrote: post #37
bobthepanda wrote: post #28

Earlier quoted context omitted.

From the article: > But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed…

You have a sample size of 1. Acknowledging that doesn't suddenly make the sample evidence of anything, good or bad :P. Almost all the miles driven are going to be in near-ideal circumstances (daylight, no rain, good road surface, driver familiar with normal road traffic conditions and drives the route regularly). I have nearly no insight into the uber death, but I gather it happened at night. It could easily be that…

The singular of data is an anecdote, as they say.

> It could easily be that humans are also an order of magnitude more dangerous at night.

Given that the actual circumstances of the death (pedestrian crossing left to right in a large road with the car in the rightmost lane) was a case that autonomous cars should've been much more equipped to deal with, and the Uber clearly failed, it's not promising. Autonomous cars were supposed to be better than humans; this particular one does not seem to be, given that as a human I can pretty clearly see someone crossing from the far side of the road.

Re: Video suggests huge problems with Uber’s driverless car program

#160

Post by KKKKkkkk1 »

matte_black wrote: post #10

Why don’t we require software engineers who work on self driving car software to go through licensing and certification. And then, if their code results in a death, they are liable and can have their license completely revoked, and they would be unable to work on self driving cars again.

There are existing laws on the books for this. Please google negligent homicide. Licensing and certification serve a different purpose.
Post reply on HN