Video suggests huge problems with Uber’s driverless car program • Hacker News Board

Re: Video suggests huge problems with Uber’s driverless car program

#121

Post by spacehome » Fri, Mar 23, 2018, 3:54 AM UTC

mozumder wrote: post #116

spacehome wrote: post #113

Earlier quoted context omitted.

Currently we're losing a million lives a year (about a 9/11 per day) to auto accidents. I think there's a serious argument that it's ethical to push autonomous driving onto the road before they're safer than humans if that would help them debug the software faster and advance to superhuman safety levels even a day sooner.

Why would you want to increase the death rate by pushing more autonomous vehicles onto the road?

Because doing so will cause issues (like this one) to be found and fixed faster. When enough bugs are fixed that the autonomous drivers are safer than the meatbag drivers, then the death rate will start to decrease. The argument is that you can trade more deaths today for a sooner decrease later. Under certain assumptions, you'll save net lives.

Re: Video suggests huge problems with Uber’s driverless car program

#122

Post by colatkinson » Fri, Mar 23, 2018, 3:56 AM UTC

PinguTS wrote: post #48

superfrank wrote: post #39

Earlier quoted context omitted.

- Expecting engineers to always write perfect code is insane. Mistakes happen. - If bad code makes it into production, that is a systemic failure not an individual one (Why didn't the bug get caught in code review, QA, etc.) - No one is going to want to work on a project where a single failure can taint their career. - What if I use a 3rd party lib and that is where the bug is. Who is at fault then? What if the code…

There is ISO26262 for automotive about safety. Yes, there are reviews, QA, and all of that. So, yes, there is no single person responsible (exceptions apply). But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it. That is the reason, why certifications are for. The same rules apply for medical and other areas.

> But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it.

Wait, what? That goes against one of the core benefits of open source software--that having many eyes on a problem decreases the risk of bugs. I'm willing to bet that if Uber had to implement their own machine learning/vision libraries from the ground up, there would be significantly more issues.

Re: Video suggests huge problems with Uber’s driverless car program

#123

Post by iClaudiusX » Fri, Mar 23, 2018, 3:57 AM UTC

pdkl95 wrote: post #99

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1]. > To get to that better software faster we should deploy the first 1000 cars asap. Is that admitting that they do not have t…

I get the feeling based on comments here that there is a severe lack of ethical and critical thinking among engineers and developers. I recognize that this is only a vocal minority but the constant mantra of "move fast and break things", where getting rich at any cost is seen as a virtue, has made me extremely disillusioned with this brand of startup culture. Doubly so when people are trading stock tips on how to profit from tragedy by supporting the worst actors in the field.

Re: Video suggests huge problems with Uber’s driverless car program

#124

Post by wdr1 » Fri, Mar 23, 2018, 3:59 AM UTC

pdkl95 wrote: post #99

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1]. > To get to that better software faster we should deploy the first 1000 cars asap. Is that admitting that they do not have t…

Is it better for self driving cars to have a flawless record & low adoption, or to have a 100x improvement over human drivers & broad adoption?

Creating the perfect self-driving car, with redundant systems, safety everything & so on will certainly help its safety records.

But it will also drive up the cost.

And put it out of reach for a lot of people.

If the goal is to save lives, the bar self-driving cars should be held to is what humans do driving today, not perfection.

Re: Video suggests huge problems with Uber’s driverless car program

#125

Post by telchar » Fri, Mar 23, 2018, 3:59 AM UTC

URSpider94 wrote: post #108

femto wrote: post #43

Earlier quoted context omitted.

I think there is an argument for removing this ambiguity by making self driving cars automatically liable for all personal injury, whether the person be inside or outside the car. The only exception would be if the tester could prove intent by the other party, keeping in mind that a self-driving car will have extensive logs of its environment to use in such a defence. Supporting arguments include: 1) The physics of d…

Your concept that there are no accidents, only oversights, is not correct. Say I’m crossing the street and I don’t see your car coming, because I look the wrong way or I’m distracted, or my view is blocked by the sun. You may not be able to avoid hitting me. Additionally, if we train self-driving cars to always give way to pedestrians who even look like they might cross the street, they’re going to have a heck of a t…

You're suggesting that self-driving cars may have to drive slowly and carefully in cities. I don't really see a problem with that. Pedestrians aren't frequently in the immediate vicinity of roads with limits higher than 35mph in my experience. Any human driver has to drive cautiously in the vicinity of pedestrians anyway, if they don't want to risk manslaughter. I'd want SDVs to do that too.

Re: Video suggests huge problems with Uber’s driverless car program

#126

Post by femto » Fri, Mar 23, 2018, 3:59 AM UTC

URSpider94 wrote: post #108

femto wrote: post #43

Earlier quoted context omitted.

I think there is an argument for removing this ambiguity by making self driving cars automatically liable for all personal injury, whether the person be inside or outside the car. The only exception would be if the tester could prove intent by the other party, keeping in mind that a self-driving car will have extensive logs of its environment to use in such a defence. Supporting arguments include: 1) The physics of d…

Your concept that there are no accidents, only oversights, is not correct. Say I’m crossing the street and I don’t see your car coming, because I look the wrong way or I’m distracted, or my view is blocked by the sun. You may not be able to avoid hitting me. Additionally, if we train self-driving cars to always give way to pedestrians who even look like they might cross the street, they’re going to have a heck of a t…

I think your first paragraph is written from the perspective of a human driver. The machine can predict based on physics. If anything a person who does not see a car will be more predictable, not less, as they will tend to be moving uniformly and not taking evasive action.

Taking into account optimal (computer driven) car stopping distances, the maximum acceleration of a person and typical pedestrian densities, I don't think getting though cities would be an issue, especially outside the CBD. Even in areas with very high pedestrian traffic cars will be able to get though. In support of this argument, I offer today's "shared" pedestrian zones, where cars and people mix. Pedestrians have right of way, cars are limited to 10km/h, but the cars manage to get though without injuring anyone. Cars would naturally do high speeds on main roads with low pedestrian density and lower speeds (with very short stopping distance) at high pedestrian density.

Why should children have a propensity to intentionally jump in front cars above and beyond anyone else? That's bias.

If anyone intentionally jumped in front of a car then it would be covered by the exemption that I proposed: that the car would not be liable if intent could be proved. Based on the car's sensor logs it would be pretty easy to prove that someone intended to get hit. If the car managed to stop and the person ran away it would then be relatively easy to track that person down based on the logs and charge them with a crime. In any case, I think that that is a hypothetical situation which is unlikely to occur. For the vast majority of people self preservation would trump the desire to cause trouble by putting oneself in a terrifying and life-threatening situation, so I think it would be negligibly rare.

Re: Video suggests huge problems with Uber’s driverless car program

#127

Post by reedx8 » Fri, Mar 23, 2018, 4:01 AM UTC

JohnJamesRambo wrote: post #62

"Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States." Wow there goes that "safer than human drivers" argument.

You can't just look at Uber to make a sweeping conclusion of all autonomous cars on the road. How many miles has Tesla, VW, Volvo, Waymo, Google, Ford, and Apple have?

Re: Video suggests huge problems with Uber’s driverless car program

#128

Post by powercf » Fri, Mar 23, 2018, 4:06 AM UTC

URSpider94 wrote: post #108

femto wrote: post #43

Earlier quoted context omitted.

I think there is an argument for removing this ambiguity by making self driving cars automatically liable for all personal injury, whether the person be inside or outside the car. The only exception would be if the tester could prove intent by the other party, keeping in mind that a self-driving car will have extensive logs of its environment to use in such a defence. Supporting arguments include: 1) The physics of d…

Your concept that there are no accidents, only oversights, is not correct. Say I’m crossing the street and I don’t see your car coming, because I look the wrong way or I’m distracted, or my view is blocked by the sun. You may not be able to avoid hitting me. Additionally, if we train self-driving cars to always give way to pedestrians who even look like they might cross the street, they’re going to have a heck of a t…

> Your concept that there are no accidents, only oversights, is not correct. Say I’m crossing the street and I don’t see your car coming, because I look the wrong way or I’m distracted, or my view is blocked by the sun. You may not be able to avoid hitting me

Solvable by reducing speed sufficiently. It's reasonable to expect the car to avoid a man travelling at 10kph from any off-road blind-spot, and a car travelling at 100kph from any on-road blind-spot.

> Additionally, if we train self-driving cars to always give way to pedestrians who even look like they might cross the street, they’re going to have a heck of a time getting through cities. Kids are going to learn that they can trigger a squealing emergency stop by lunging towards the curb - great fun!

If you sprint to the curb in front of traffic today, drivers will stop/swerve. Almost certainly illegal too.

My interpretation of the parent post is that more responsibility can be put on the car to avoid accidents, than is currently the case today. Hence greatly increasing road safety. It sounds great!

Re: Video suggests huge problems with Uber’s driverless car program

#129

Post by davidgay » Fri, Mar 23, 2018, 4:08 AM UTC

superfrank wrote: post #39

matte_black wrote: post #10

Why don’t we require software engineers who work on self driving car software to go through licensing and certification. And then, if their code results in a death, they are liable and can have their license completely revoked, and they would be unable to work on self driving cars again.

- Expecting engineers to always write perfect code is insane. Mistakes happen. - If bad code makes it into production, that is a systemic failure not an individual one (Why didn't the bug get caught in code review, QA, etc.) - No one is going to want to work on a project where a single failure can taint their career. - What if I use a 3rd party lib and that is where the bug is. Who is at fault then? What if the code…

> - Expecting engineers to always write perfect code is insane. Mistakes happen.

In safety-critical fields, setting a much higher quality bar than the regular 'it seems to works, the tests pass' seems perfectly rational to me. We can now write provably-correct C compilers (CompCert) and OS kernels (SeL4). There's no excuse for not putting similar levels of effort[0] into something as safety-critical as self-driving cars.

[0]: Note that I'm not advocating for "provably-correct self-driving car software" (that may not be the right approach, as a formal spec is likely unrealisable), but find the argument that "it's ok to write buggy spreadsheets, so it's ok to write buggy self-driving cars" to be morally unacceptable.

Re: Video suggests huge problems with Uber’s driverless car program

#130

Post by telchar » Fri, Mar 23, 2018, 4:08 AM UTC

aylons wrote: post #8

"Move fast and break things" is exactly the opposite of what a responsible driver must do.

I've been joking for at least a year that Uber's motto is "move fast and break people". I'm saddened that this has come to pass but not surprised.