Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

US probes Tesla's Full Self-Driving software after fatal crash

reuters.com

Post reply on HN
Print view View original

551–560 of 1001 posts

Re: US probes Tesla's Full Self-Driving software after fatal crash

#551

Post by awongh »

Arainach wrote: post #241
AlchemistCamp wrote: post #226

The interesting question is how good self-driving has to be before people tolerate it. It's clear that having half the casualty rate per distance traveled of the median human driver isn't acceptable. How about a quarter? Or a tenth? Accidents caused by human drivers are one of the largest causes of injury and death, but they're not newsworthy the way an accident involving automated driving is. It's all too easy to se…

This is about lying to the public and stoking false expectations for years. If it's "fully self driving" Tesla should be liable for when its vehicles kill people. If it's not fully self driving and Tesla keeps using that name in all its marketing, regardless of any fine print, then Tesla should be liable for people acting as though their cars could FULLY self drive and be sued accordingly. You don't get to lie just b…

Also force other auto makers to be liable when their over-tall SUVs cause more deaths than sedan type cars.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#552

Post by kergonath »

viraptor wrote: post #511
TheCleric wrote: post #440

Earlier quoted context omitted.

Good. If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally.

That's a dangerous line and I don't think it's correct. Software I write shouldn't be relied on in critical situations. If someone makes that decision then it's on them not on me. The line should be where a person tells others that they can rely on the software with their lives - as in the integrator for the end product. Even if I was working on the software for self driving, the same thing would apply - if I wrote s…

It’s not that complicated or outlandish. That’s how most engineering fields work. If a building collapses because of design flaws, then the builders and architects can be held responsible. Hell, if a car crashes because of a design or assembly flaw, the manufacturer is held responsible. Why should self-driving software be any different?

If the software is not reliable enough, then don’t use it in a context where it could kill people.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#553

Post by kergonath »

everforward wrote: post #519
ryandrake wrote: post #485

Earlier quoted context omitted.

Your take is understandable and not surprising on a site full of software developers. Somehow, the general software industry has ingrained this pessimistic and fatalistic dogma that says bugs are inevitable and there’s nothing you can do to prevent them. Since everyone believes it, it is a self-fulfilling prophecy and we just accept it as some kind of law of nature. Holding software developers (or their companies) li…

It is true of every field I can think of. Food gets salmonella and what not frequently. Surgeons forget sponges inside of people (and worse). Truckers run over cars. Manufacturers miss some failures in QA. Literally everywhere else, we accept that the costs of 100% safety are just unreasonably high. People would rather have a mostly safe device for $1 than a definitely safe one for $5. No one wants to pay to have eve…

> Literally everywhere else, we accept that the costs of 100% safety are just unreasonably high.

Yes, but also in none of these situations would the consumer/customer/patient be held responsible. I don’t expect a system to be perfect, but I won’t accept any liability if it malfunctions as I use it the way it is intended. And even worse, I would not accept that the designers evade their responsibilities if it kills someone I know.

As the other poster said, I am happy to consider it safe enough the day the company accepts to own its issues and the associated responsibility.

> No one wants to pay for a formally verified Facebook or DoorDash.

This is untenable. Does nobody want a formally verified avionics system in their airliner, either?

Re: US probes Tesla's Full Self-Driving software after fatal crash

#554

Post by kergonath »

ukuina wrote: post #520
thunky wrote: post #502

Earlier quoted context omitted.

You can go to prison or die for being a bad driver, yet people choose to drive.

Systems evolve to handle such liability: Drivers pass theory and practical tests to get licensed to drive (and periodically thereafter), and an insurance framework that gauges your risk-level and charges you accordingly.

Requiring formal licensing and possibly insurance for developers working on life-critical systems is not that outlandish. On the contrary, that is already the case in serious engineering fields.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#555

Post by innocentoldguy »

massysett wrote: post #64

"Tesla says on its website its FSD software in on-road vehicles requires active driver supervision and does not make vehicles autonomous." Despite it being called "Full Self-Driving." Tesla should be sued out of existence.

It's called "Full Self-Driving (Supervised) Beta" and you agree that you understand that you have to pay attention and are responsible for the safety of the car before you turn it on.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#556

Post by elric »

viraptor wrote: post #533
presentation wrote: post #529

Earlier quoted context omitted.

To be fair maybe the software you write shouldn’t be relied on in critical situations but in this case the only place this software could be used in are critical situations

Ultimately - yes. But as I mentioned, the fact it's sold as ready for critical situations doesn't mean the developers thought/said it's ready.

I think it should be fairly obvious that it's not the individual developers who are responsible/liable. In critical systems there is a whole chain of liability. That one guy in Nebraska who thanklessly maintains some open source lib that BigCorp is using in their car should obviously not be liable.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#557

Post by gmueckl »

viraptor wrote: post #533
presentation wrote: post #529

Earlier quoted context omitted.

To be fair maybe the software you write shouldn’t be relied on in critical situations but in this case the only place this software could be used in are critical situations

Ultimately - yes. But as I mentioned, the fact it's sold as ready for critical situations doesn't mean the developers thought/said it's ready.

But someone slapped that label on it and made a pinky promise that it's true. That person needs to accept liability if things go wrong. If person A is loud and clear that something isn't ready, but person B tells the customer otherwise, B is at fault.

Look, there are well established procedures in a lot of industries where products are relied on to keep people safe. They all require quite rigorous development and certification processes and sneaking untested alpha quality software through such a process would be actively malicious and quite possibly criminal in and of itself, at least in some industries.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#558

Post by TheAlchemist »

Tesla released a promotional video in 2016 saying that with FSD a human driver is not necessary and that "The person in the driver's seat is only there for legal reasons". The video was staged as we've learned in 2022.

2016 folks... Even with today's FSD which is several orders of magnitude better than the one in the video, you would still probably have a serious accident within a week (and I'm being generous here) if you didn't seat in the driver's seat.

How Trevor Milton got sentenced for fraud and the people responsible for this were not is a mystery to me.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#559

Post by tsimionescu »

ryandrake wrote: post #485
dmix wrote: post #468

Earlier quoted context omitted.

Drug companies and the FDA (circa 1906) play a very dangerous and delicate dance all the time releasing new drugs to the public. But for over a century now we've managed to figure it out without holding pharma companies criminally liable for every death. > If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally. Easy to type those words on the internet t…

Your take is understandable and not surprising on a site full of software developers. Somehow, the general software industry has ingrained this pessimistic and fatalistic dogma that says bugs are inevitable and there’s nothing you can do to prevent them. Since everyone believes it, it is a self-fulfilling prophecy and we just accept it as some kind of law of nature. Holding software developers (or their companies) li…

> And why don’t they spend 90% of their time testing and verifying and proving their software has no defects? Because defects are inevitable and they’re not held accountable for them!

For a huge part of the industry, the reason is entirely different. It is because software that mostly works today but has defects is much more valuable than software that always works and has no defects 10 years from now. Extremely well informed business customers will pay for delivering a buggy feature today rather than wait two more months for a comprehensively tested feature. This is the reality of the majority of the industry: consumers care little about bugs (below some defect rate) and care far more about timeliness.

This of course doesn't apply to critical systems like automatic drivers or medical devices. But the vast majority of the industry is not building these types of systems.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#560

Post by viraptor »

gmueckl wrote: post #557
viraptor wrote: post #533

Earlier quoted context omitted.

Ultimately - yes. But as I mentioned, the fact it's sold as ready for critical situations doesn't mean the developers thought/said it's ready.

But someone slapped that label on it and made a pinky promise that it's true. That person needs to accept liability if things go wrong. If person A is loud and clear that something isn't ready, but person B tells the customer otherwise, B is at fault. Look, there are well established procedures in a lot of industries where products are relied on to keep people safe. They all require quite rigorous development and cer…

This is the beginning of the thread https://news.ycombinator.com/item?id=41891164

You're in violent agreement with me ;)

Post reply on HN