US probes Tesla's Full Self-Driving software after fatal crash • Hacker News Board

Re: US probes Tesla's Full Self-Driving software after fatal crash

#511

Post by viraptor » Sun, Oct 20, 2024, 3:06 AM UTC

TheCleric wrote: post #440

bdcravens wrote: post #431

Earlier quoted context omitted.

The liability for killing someone can include prison time.

Good. If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally.

That's a dangerous line and I don't think it's correct. Software I write shouldn't be relied on in critical situations. If someone makes that decision then it's on them not on me.

The line should be where a person tells others that they can rely on the software with their lives - as in the integrator for the end product. Even if I was working on the software for self driving, the same thing would apply - if I wrote some alpha level stuff for the internal demonstration and some manager decided "good enough, ship it", they should be liable for that decision. (Because I wouldn't be able to stop them / may have already left by then)

Re: US probes Tesla's Full Self-Driving software after fatal crash

#512

Post by bastawhiz » Sun, Oct 20, 2024, 3:07 AM UTC

absoflutely wrote: post #500

bastawhiz wrote: post #494

Earlier quoted context omitted.

Autopilot has not been good. I have a cabin four hours from my home and I've used autopilot for long stretches on the highway. Some of the problems: - Certain exits are not detected as such and the car violently veers right before returning to the lane. I simply can't believe they don't have telemetry to remedy this. - Sometimes the GPS becomes miscalibrated. This makes the car think I'm taking an exit when I'm not,…

I'll add that it randomly brakes hard on the interstate because it thinks the speed limit drops to 45. There aren't speed limit signs anywhere nearby on different roads that it could be mistakenly reading either.

I noticed that this happens when the triangle on the map is slightly offset from the road, which I've attributed to miscalibrated GPS. It happens consistently when I'm in the right lane and pass an exit when the triangle is ever so slightly misaligned.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#513

Post by bastawhiz » Sun, Oct 20, 2024, 3:09 AM UTC

Uzza wrote: post #415

bastawhiz wrote: post #274

Earlier quoted context omitted.

Is Tesla required to report system failures or the vehicle damaging itself? How do we know they're not optimizing for the benchmark (what they're legally required to report)?

All manufacturers have for some time been required by regulators to report any accident where an autonomous or partially autonomous system was active within 30 seconds of an accident.

My question is better rephrased as "what is legally considered an accident that needs to be reported?" If the car scrapes a barricade or curbs it hard but the airbags don't deploy and the car doesn't sense the damage, clearly they don't. There's a wide spectrum of issues up to the point where someone is injured or another car is damaged.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#514

Post by IshKebab » Sun, Oct 20, 2024, 3:13 AM UTC

macintux wrote: post #219

alexjplant wrote: post #216

> The collision happened because the sun was in the Tesla driver's eyes, so the Tesla driver was not charged, said Raul Garcia, public information officer for the department. Am I missing something or is this the gross miscarriage of justice that it sounds like? The driver could afford a $40k vehicle but not $20 polarized shades from Amazon? Negligence is negligence.

I have no idea what the conditions were like for this incident, but I’ve blown through a 4-way stop sign when the sun was setting. There’s only so much sunglasses can do.

I know right? Once I got something in my eye so I couldn't see at all, but I decided that since I couldn't do anything about it the best thing was to keep driving. I killed a few pedestrians but... eh, what was I going to do?

Re: US probes Tesla's Full Self-Driving software after fatal crash

#515

Post by viraptor » Sun, Oct 20, 2024, 3:13 AM UTC

ryandrake wrote: post #485

dmix wrote: post #468

Earlier quoted context omitted.

Drug companies and the FDA (circa 1906) play a very dangerous and delicate dance all the time releasing new drugs to the public. But for over a century now we've managed to figure it out without holding pharma companies criminally liable for every death. > If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally. Easy to type those words on the internet t…

Your take is understandable and not surprising on a site full of software developers. Somehow, the general software industry has ingrained this pessimistic and fatalistic dogma that says bugs are inevitable and there’s nothing you can do to prevent them. Since everyone believes it, it is a self-fulfilling prophecy and we just accept it as some kind of law of nature. Holding software developers (or their companies) li…

> that says bugs are inevitable and there’s nothing you can do to prevent them

I don't think people believe this as such. It may be the short way to write it, but actually what devs mean is "bugs are inevitable at the funding/time available". I often say "bugs are inevitable" when it practice it means "you're not going to pay a team for formal specification, validated implementation and enough reliable hardware".

Which business will agree to making the process 5x longer and require extra people? Especially if they're not forced there by regulation or potential liability?

Re: US probes Tesla's Full Self-Driving software after fatal crash

#516

Post by bastawhiz » Sun, Oct 20, 2024, 3:15 AM UTC

browningstreet wrote: post #510

bastawhiz wrote: post #257

Lots of people are asking how good the self driving has to be before we tolerate it. I got a one month free trial of FSD and turned it off after two weeks. Quite simply: it's dangerous. - It failed with a cryptic system error while driving - It started making a left turn far too early that would have scraped the left side of the car on a sign. I had to manually intervene. - In my opinion, the default setting accelera…

Was this the last version, or the version released today? I’ve been pretty skeptical of FSD and didn’t use the last version much. Today I used the latest test version, enabled yesterday, and rode around SF, to and from GGP, and it did really well. Waymo well? Almost. But whereas I haven’t ridden Waymo on the highway yet, FSD got me from Hunters Point to the east bay with no disruptions. The biggest improvement I noti…

I'm clearly not using the FSD today because I refused to complete my free trial of it a few months ago. The post of mine that you're responding to doesn't mention my troubles with Autopilot, which I highly doubt are addressed by today's update (see my other comment for a list of problems). They need to really, really prove to me that Autopilot is working reliably before I'd even consider accepting another free trial of FSD, which I doubt they'd do anyway.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#517

Post by ilyagr » Sun, Oct 20, 2024, 3:22 AM UTC

mrpippy wrote: post #426

Arainach wrote: post #241

Earlier quoted context omitted.

This is about lying to the public and stoking false expectations for years. If it's "fully self driving" Tesla should be liable for when its vehicles kill people. If it's not fully self driving and Tesla keeps using that name in all its marketing, regardless of any fine print, then Tesla should be liable for people acting as though their cars could FULLY self drive and be sued accordingly. You don't get to lie just b…

Tesla officially renamed it to “Full Self Driving (supervised)” a few months ago, previously it was “Full Self Driving (beta)” Both names are ridiculous, for different reasons. Nothing called a “beta” should be tested on public roads without a trained employee supervising it (i.e. being paid to pay attention). And of course it was not “full”, it always required supervision. And “Full Self Driving (supervised)” is an…

I hope they soon rename it into "Fully Supervised Driving".

Re: US probes Tesla's Full Self-Driving software after fatal crash

#518

Post by rootusrootus » Sun, Oct 20, 2024, 3:22 AM UTC

I'm on my second free FSD trial, just started for me today. Gave it another shot, and it seems largely similar to the last free trial they gave. Fun party trick, surprisingly good, right up until it's not. A hallmark of AI everywhere, is how great it is and just how abruptly and catastrophically it fails occasionally.

Please, if you're going to try it, keep both hands on the wheel and your foot ready for the brake. When it goes off the rails, it usually does so in surprising ways with little warning and little time to correct. And since it's so good much of the time, you can get lulled into complacence.

I never really understand the comments from people who think it's the greatest thing ever and makes their drive less stressful. Does the opposite for me. Entertaining but exhausting to supervise.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#519

Post by everforward » Sun, Oct 20, 2024, 3:30 AM UTC

ryandrake wrote: post #485

dmix wrote: post #468

Earlier quoted context omitted.

Drug companies and the FDA (circa 1906) play a very dangerous and delicate dance all the time releasing new drugs to the public. But for over a century now we've managed to figure it out without holding pharma companies criminally liable for every death. > If you write software that people rely on with their lives, and it fails, you should be held liable for that criminally. Easy to type those words on the internet t…

Your take is understandable and not surprising on a site full of software developers. Somehow, the general software industry has ingrained this pessimistic and fatalistic dogma that says bugs are inevitable and there’s nothing you can do to prevent them. Since everyone believes it, it is a self-fulfilling prophecy and we just accept it as some kind of law of nature. Holding software developers (or their companies) li…

It is true of every field I can think of. Food gets salmonella and what not frequently. Surgeons forget sponges inside of people (and worse). Truckers run over cars. Manufacturers miss some failures in QA.

Literally everywhere else, we accept that the costs of 100% safety are just unreasonably high. People would rather have a mostly safe device for $1 than a definitely safe one for $5. No one wants to pay to have every head of lettuce tested for E Coli, or truckers to drive at 10mph so they can’t kill anyone.

Software isn’t different. For the vast majority of applications where the costs of failure are low to none, people want it to be free and rapidly iterated on even if it fails. No one wants to pay for a formally verified Facebook or DoorDash.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#520

Post by ukuina » Sun, Oct 20, 2024, 3:35 AM UTC

thunky wrote: post #502

jalapenos wrote: post #481

Earlier quoted context omitted.

What a laugh, would you take that deal? Upside: you get paid a 200k salary, if all your code works perfectly. Downside: if it doesn't, you go to prison. The users aren't compelled to use it. They can choose not to. They get to choose their own risks. The internet is a gold mine of creatively moronic opinions.

You can go to prison or die for being a bad driver, yet people choose to drive.

Systems evolve to handle such liability: Drivers pass theory and practical tests to get licensed to drive (and periodically thereafter), and an insurance framework that gauges your risk-level and charges you accordingly.