Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Tragic Loss

teslamotors.com

Post reply on HN
Print view View original

351–360 of 511 posts

Re: A Tragic Loss

#351

Post by mthoms »

jsprogrammer wrote: post #300
YeGoblynQueenne wrote: post #287

Earlier quoted context omitted.

>> The point is that humanity is not a herd to be taken as a whole; That's a great point but the truth is that we already treat human lives as a resource to be spent: for instance, we allow planes to fly and automobiles to be driven even though their safety is not perfect. We allow that, out of say a million flights, one plane will fatally crash killing usually everybody on board (I don't know what the real number is…

We actually don't allow planes to crash. Crashes are a result of human error or catastrophic failure. When a crash occurs, the cause is investigated so that another crash never happens for the same reason. If a defect is ever found, all affected planes are grounded until they have been fixed. I expect Autopilot to be disabled soon. Edit: >So basically we accept that in order for each of us to fly to our destination,…

There are many factors that can never be anticipated and we explicitly accept that they still exist.

>No plane will take off if there is a calculated chance that it could crash.

I don't get this comment. If there are statistics that prove one in every X million planes crash, then every plane taking off has a 1 in X million chance of crashing.

There is actually a very precise calculated chance that it could crash (but air travellers generally accept it).

Re: A Tragic Loss

#352

Post by jfrisby »

archagon wrote: post #345
xoa wrote: post #327

Earlier quoted context omitted.

> How do you counter a utilitarian argument without humanizing the situation? Have you considered that perhaps there is no argument to be countered? That your entire foundation of "humanizing" the situation is foolish and off base? Alternatively, you yourself blithely dismissed the fact that your "humanizing" easily works both ways, in fact more easily works the other way: many of us have had friends or loved ones ki…

Nowhere in my original comment did I make the case that self-driving cars should be pre-emptively banned, or that I personally should have any say in the matter. Believe me — I am all for the legal system (and "society") determining the future of this technology. But at this point in time, I am not convinced that people against this future would, in fact, be out-voted. And if that turns out to be the case, it would d…

If you are any example, then the reservations are unreasonable and there's plenty of precedent for ignoring them.

We didn't ban vaccines because a handful of people are harmed, and we're not reconsidering their use because a bunch of mathematically illiterate loons think they're more dangerous than the alternative.

Frankly, there's more reason to ban PEOPLE from being able to drive a car than to ban AIs from driving them -- if only because AIs can be improved and people cannot.

All the ignorance of probability, moral philosophy, economics and law will not change the fact that the posture you are arguing from is one of preserving a status quo that costs dozens of people their lives every day.

Re: A Tragic Loss

#353

Post by ceejayoz »

takeda wrote: post #337
pacificmint wrote: post #299

Earlier quoted context omitted.

> Based on what? Human drivers are very bad. We kill over 32,000 people a year (and that's a 60 year low, just a few years ago it was over 40,000). So if self driving cars are just a little better than humans, they will save many lives. If they are a lot better than us, they will save many more.

Human programmers are even worse we make 15-50 bugs per 1000 lines of code. Also don't forget that while each of us is somewhat unique in terms of driving, the cars are not, so a bug in a car's software can amplify issues even further. There are issues also how cars will behave when they will have to interact with other self driving cars. Those problems are already difficult when writing distributed applications and…

> Human programmers are even worse we make 15-50 bugs per 1000 lines of code.

Only when that's deemed acceptable.

http://www.fastcompany.com/28121/they-write-right-stuff

> But how much work the software does is not what makes it remarkable. What makes it remarkable is how well the software works. This software never crashes. It never needs to be re-booted. This software is bug-free. It is perfect, as perfect as human beings have achieved. Consider these stats : the last three versions of the program — each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.

Re: A Tragic Loss

#354

Post by jfrisby »

ModernMech wrote: post #249
archagon wrote: post #221

This is on the road to being off topic, but still relevant given some of the commentary in this thread: It makes me a bit sad that the political zeitgeist in the tech community is leaning towards "acceptable losses" when it comes to accidents in automated cars, to the point of pre-emptively expressing disdain at ordinary people reacting negatively to such news. I sense it's going to become harder and harder for us to…

> It makes me a bit sad that the political zeitgeist in the tech community is leaning towards "acceptable losses" when it comes to accidents in automated cars Yes, this is another thing that makes me so angry about this situation. Back in the day, I worked on a project to give autonomy to power wheelchair systems. In order to get it to market, we had to go through FDA approval, which involved testing the equipment in…

Except that the car wasn't autonomous here.

The human behind the wheel had been explicitly instructed to keep hands on the wheel and feet on the pedals -- and that it was his responsibility to override the machine's judgement if circumstances warranted.

Re: A Tragic Loss

#355

Post by archagon »

jfrisby wrote: post #352
archagon wrote: post #345

Earlier quoted context omitted.

Nowhere in my original comment did I make the case that self-driving cars should be pre-emptively banned, or that I personally should have any say in the matter. Believe me — I am all for the legal system (and "society") determining the future of this technology. But at this point in time, I am not convinced that people against this future would, in fact, be out-voted. And if that turns out to be the case, it would d…

If you are any example, then the reservations are unreasonable and there's plenty of precedent for ignoring them. We didn't ban vaccines because a handful of people are harmed, and we're not reconsidering their use because a bunch of mathematically illiterate loons think they're more dangerous than the alternative. Frankly, there's more reason to ban PEOPLE from being able to drive a car than to ban AIs from driving…

Sigh. OK, look. Your inflammatory rhetoric is exactly the sort of thing I was trying to warn against, but I guess self-driving cars are part of the political shitpile now, too. It's a shame that important issues like this seemingly cannot be discussed without resorting to tribalism and name-calling.

We're not in your imaginary super-safe self-driving future yet, and if lives are to be carelessly sacrificed as stepping stones on the way there — because Tesla decided to beta test some dangerous software on actual, real-life roads — it is never going to happen. No matter how much of a utilitarian benefit there might be in the long run.

Re: A Tragic Loss

#356

Post by thaumasiotes »

jsprogrammer wrote: post #347
YeGoblynQueenne wrote: post #338

Earlier quoted context omitted.

>> If there is a known chance that a component of a plane might fail and cause a crash, it will not take off. There is no component of any machine that is 100% guaranteed to not fail under any circumstances. What the bloody hell are you talking about?

Known, calculable circumstances. If a component may fail, then redundancies are added so that failure will not result in a crash.

You just keep digging. All adding redundancy can do is lower the chance of a crash.

> If a component may fail

As has already been pointed out to you, all components may fail, including any that you add to compensate for the potential failure of other components.

An engineer should know this by instinct.

Re: A Tragic Loss

#357

Post by Kalium »

archagon wrote: post #336
jfrisby wrote: post #329

Earlier quoted context omitted.

Not so much "a hoot" as "morally misguided", and "void of empathy for people outside his/her monkey-sphere."

Putting aside the fact that many ordinary people would find your perspective lacking humanity and hilariously on the opposite side of their own moral compass: I really didn't offer much other than an opposing perspective and a call to be more moderate. This will come up again and again, and it would pay to have a bit of empathy here.

I have empathy. I have empathy for the friends and family of those we cannot save. I also have empathy for the reduced number of people that will suffer losses of friends and loved ones as we reduce harm to the human herd.

What should I do? If you think jfrisby and myself lack empathy, what do you think empathy looks like and how do you think it would change our moral stances?

Is this an academic question to you? Or do you actually have these questions weighing on your conscience as a result of things you've done in the course of your work?

Re: A Tragic Loss

#359

Post by AceyMan »

emp_zealoth wrote: post #245
AceyMan wrote: post #234

Earlier quoted context omitted.

The Air France 447 accident is a better fitting example of pitfalls that may obtain in complex "humans-with-automation" types of systems. There, automation lowered both the standard for situational awareness and fundamental stick and rudder skills. Then, when a quirky corner case happened, the pilots did all manner of wrong on the problem: so much so, they amplified a condition from "mostly harmless" to fatal for all…

I heard it was the Airbus weirdness of steering setup that noticeably added to the problems (Separate, disjointed joysticks) One pilot pulled up as hard as he could while the other one thought he was pushing down, making the confusion this much worse

That's true, but was well known (and trained on), so I'd categorize that domain as "How the machine responds when you're hands are on the controls," which is nearly a synonym for "stick and rudder skills" category I cited.

Sure, to nearly every pilot that behavior is wacky, but it shouldn't have been a surprise for more than an instant to pilots who were "operating as designed."

It seems there's no free lunch: when skills atrophy as a natural response to helpful automation it requires advancement in some other skills, should the goal of an ever improving error (accident) rate be achieved.

Re: A Tragic Loss

#360

Post by jfrisby »

archagon wrote: post #336
jfrisby wrote: post #329

Earlier quoted context omitted.

Not so much "a hoot" as "morally misguided", and "void of empathy for people outside his/her monkey-sphere."

Putting aside the fact that many ordinary people would find your perspective lacking humanity and hilariously on the opposite side of their own moral compass: I really didn't offer much other than an opposing perspective and a call to be more moderate. This will come up again and again, and it would pay to have a bit of empathy here.

I have empathy for the ~33,000 people that die every year because humans are terrible drivers.

You do not. You want to take a moral posture that condemns thousands of people to needless deaths because you're so caught up in emotion you can't see the harm your position would cause.

Wallowing in your own emotions is selfish and immoral when it costs actual lives.

Your entire premise is based on the false assumption that this system was making all the decisions. Tesla's "autopilot" does not override the driver. It does not replace the driver (yet). That's explicitly because everyone involved knows it's not ready for that responsibility yet, no matter how much they've tested it. Tesla knows they cannot possibly anticipate everything -- the whole point of this "public beta" is to gather more information and help them make a better, safer system that eventually is trustworthy enough to consistently make decisions better than people.

You posit that human decision-makers behind the wheel are an important thing we should be wary of giving up but you have ignored one simple fact: There WAS a human behind the wheel, expressly charged -- both by law and by Tesla -- with maintaining control of the vehicle.

The driver had the responsibility of hitting the brakes, and did not do so. That's an error that plays out dozens of times per day without AI drivers being involved.

But instead of recognizing that this is a case of not just a piece of software failing, but also of a human failing -- and instead of recognizing the cost of lost and shattered lives borne by society every day -- you resort to ill-conceived hand-wringing in an attempt to convince others to follow you down a path that would continue condemning untold numbers of people to needless deaths.

Post reply on HN