Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

arstechnica.com

Post reply on HN
Print view View original

541–546 of 546 posts

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#541

Post by buvanshak »

JoshCole wrote: post #540
buvanshak wrote: post #539

Earlier quoted context omitted.

>I never disagreed that there was potential for the safety to be negatively impacted with every update... The point you are missing is that while update might be slightly enhance safety, the negative, unpredictable impact might be catastrophic. because one is planned and other one is not.

No, the point you're pretending I'm missing is that there is this potential. You quoted text in which I explicitly acknowledge the danger of updates. Read it. Also, read further down where I explicitly ask that I not be taken the wrong way, because good points were made. My acknowledgement only make sense if I agree that there is some level of danger in each update. That is why you're addressing a straw man. I feel l…

>No, the point you're pretending I'm missing is that there is this potential

No. You are not missing the potential. But you do not seem to get difference in magnitude. One is incremental, reviewed safety enhancement, Other is unpredictably catastrophic.

You only seem to grasp very superficial aspects of my comments, which is why I requested to give some thought before responding. So I think there is some kind of barrier. But it is not of language. but for lack of a better word, I think it is of a lack of enough shared sensibilities..

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#542

Post by JoshCole »

buvanshak wrote: post #541
JoshCole wrote: post #540

Earlier quoted context omitted.

No, the point you're pretending I'm missing is that there is this potential. You quoted text in which I explicitly acknowledge the danger of updates. Read it. Also, read further down where I explicitly ask that I not be taken the wrong way, because good points were made. My acknowledgement only make sense if I agree that there is some level of danger in each update. That is why you're addressing a straw man. I feel l…

>No, the point you're pretending I'm missing is that there is this potential No. You are not missing the potential. But you do not seem to get difference in magnitude. One is incremental, reviewed safety enhancement, Other is unpredictably catastrophic. You only seem to grasp very superficial aspects of my comments, which is why I requested to give some thought before responding. So I think there is some kind of barr…

You're projecting that the projected future upside isn't superhuman. My comment projects that it is. We disagree on projected benefits.

Right now human driving is one of the leading causes of death. I believe that technology can eventually eliminate this as a leading cause of death. So I project a much greater potential upside. I also figure this is a matter of time and effort applied to the problem. Or in other words, there is a finite amount of time before an update brings the car to this point. This puts a ceiling on my mental tabulation of the amount of risk endured prior to achieving an extremely good end. So despite the severe risks, the limited nature of that risk allows me to rule in favor of taking the risk despite its presence.

You're assuming that I haven't pictured a sweeping update which adds the car murdering anyone who was unaware. I have! Your assumption is incorrect.

And if I was being superficial, I would have answered that yes, I'm a software developer. But its a fallacious appeal to authority.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#543

Post by dnautics »

amelius wrote: post #475
dnautics wrote: post #430

Earlier quoted context omitted.

There's a lot of sunlight between letting pharma companies run rampant and having the FDA. One could imagine private non-profit testing and qualifications standards organizations along the lines of the underwriters laboratories

And who will manage all that testing? A government entity? Because that would bring us back to the original point.

UL is not "managed" by a government.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#544

Post by prawn »

Robotbeat wrote: post #326
prawn wrote: post #294

Earlier quoted context omitted.

I'd prevent them from using the AutoPilot term, penalise them for marketing imagery that includes people driving with their hands off the wheel, and require hands to be on the wheel far more often. Until it's significantly improved, it should be a backup safety system rather than a driving and safety system.

> I'd prevent them from using the AutoPilot term I disagree. It's the same term as in aviation and does the same thing. > penalise them for marketing imagery that includes people driving with their hands off the wheel That's more reasonable.

As I understand it, aviation autopilot is generally used in open air where there are typically not frequent stationary objects the software is not designed to detect. Also usually sold in a different way to a different market.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#545

Post by InternetOfStuff »

bertil wrote: post #487
InternetOfStuff wrote: post #451

Earlier quoted context omitted.

> And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal I recently got downvoted for that…

I think you got down-voted because you misrepresented what Telsa is actually doing, which is a difficult arbitrage between: - known preventable deaths from, say, not staying in the lane aggressively enough; - possible surprises and subsequent deaths. There is an ethical conundrum (that is quite different from the trolley problem) between a known fix and an unknown possibility. If both are clearly establish and you di…

Thanks for your comment. I think you may be right.

> I think you got down-voted because you misrepresented what Telsa is actually doing, which is a difficult arbitrage between: > > - known preventable deaths from, say, not staying in the lane aggressively enough; > > - possible surprises and subsequent deaths.

I don't think I was misrepresenting anything (at least, I was trying not to). I just pointed out that behaviour-changing updates that may be harmless in, say, smartphone apps, are much more problematic in environments such as driving-assisted cars. I think this is objectively true. And I think we need to come up with mechanisms to solve these problems.

> That learning pattern (resolving unintended surprises as they happen decreasingly often) is common in software.

My argument is that changes in behaviour are (almost automatically) surprising, and thus inherently dangerous. Unless my car is truly autonomous and doesn't need my intervention, it must be predictable. Updates run the risk oof breaking htat predictability.

> Others have preferred the surprise-free and PR-friendly option of not saving the dozens of thousand of lives dying on the road at the moment.

My worry is that (potentially) people will still die, just different ones.

> being in favour of Telsa (and Waymo) taking more risk than necessary

If I'm taking you literally, that's an obviously unwise position to take ("more than necessary"). But I think I know what you meant to say: err on the side of faster learning, accepting the potential consequences. Perhaps like NASA in the 1960s.

But my argument was simply that there is a problem with frequent, gradual updates. Not that we shouldn't update (even though that's actually one option). We ought to search for solutions to this problem. I can think of several that aren't "don't update".

But claiming that the problem doesn't exist, or that those that worry about it are unreasonable, is unhelpful.

Re: NTSB: Autopilot steered Tesla car toward traffic barrier before deadly crash

#546

Post by workinthehead »

hexane360 wrote: post #529
koyote wrote: post #483

Earlier quoted context omitted.

The European road network is actually slightly larger than the US one (~6.5 million km vs ~7 million km).

Which makes the point just fine, when you compare the U.S's 326 million to the EU's 508 million.

Well, 741 million, I don't believe the road network figures were restricted to EU members only.
Post reply on HN