Tesla is ordered to turn over Autopilot data to a federal safety agency • Hacker News Board

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#81

Post by mikestew » Thu, Sep 02, 2021, 2:33 PM UTC

AshamedCaptain wrote: post #19

capableweb wrote: post #6

Earlier quoted context omitted.

> 12 crashes into vehicles with flashing lights on them doesn't sound like great success from Autopilot, gonna be honest. I'm sorry, but how can you form an opinion so quickly? 12 crashes out of what? Any number? 12 crashes out of 10000000 scenarios sound like a pretty good record, while 12 crashes out of 15 scenarios indeed doesn't sound like a great success. But what information do you have that the rest of us does…

I still remember the days when the SciFi of the day sold robotic drivers as "they don't get tired, they don't make mistakes, they are relentless in their pursuit". Instead of that, we got robots that are apparently unable to avoid crashing into large vehicles with flashing lights. Sometimes. As usual, the future never ceases to disappoint me.

"they don't get tired, they don't make mistakes, they are relentless in their pursuit".

I believe you're thinking of terminators, not automated drivers:

https://quotegeek.com/quotes-from-movies/the-terminator/1136...

Though the "will not stop, ever...", even for an emergency vehicle, might be relevant.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#82

Post by ocdtrekkie » Thu, Sep 02, 2021, 2:33 PM UTC

lftl wrote: post #53

eganist wrote: post #37

Earlier quoted context omitted.

"I'm sorry, but how can you form an opinion so quickly?" It doesn't really matter. They're stationary objects; there's absolutely no reason, good or bad, for the algorithm to determine that the correct trajectory change is into the stationary object. Humans crash into stationary objects because they're lacking attention almost 100% of the time, whether it's because they're half asleep, drunk, texting, or distracted b…

I can't find the specific comment now, but on previous autonomous driving discussions here, it's been asserted that Tesla (and many other systems) simply can't reliably detect stationary objects at speed. If the systems actually did a naive, "that looks like a stationary object, I'm going to avoid it" they'd barely be able to move in a highway setting.

If that's the case, the correct solution is to shut Autopilot and all systems like it down, and charge their developers with reckless endangerment.

A autonomous driving system which can't detect stationary obstacles is not an autonomous driving system: It's a murder machine.

If they can't do this, they shouldn't operate, full stop.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#83

Post by joelwilliamson » Thu, Sep 02, 2021, 2:35 PM UTC

jerf wrote: post #7

mhermher wrote: post #5

Earlier quoted context omitted.

The fact that it occurs so often should suggest to you that autopilot itself makes people too comfortable and not pay attention.

I, and many others, take the position that there really isn't any practical gradation between more-or-less conventional driving, maybe with light lane switching assistance or something, and full on "Level 5" autonomous driving, because the idea that human brains can watch something for hours at a time only to take instant, sub-second actions every few hours is an insane model of human cognition. I don't disagree with…

Shouldn't the objection be to anything less than Level 4, rather than Level 5? Level 4 requires that the vehicle can safely stop itself and wait for a human to take over when it is faced with a situation it can't handle.

(There's an argument that if you take away the routine parts of driving, humans won't be able to deal with anomalous situations, but that's a different argument than the attentiveness one.)

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#84

Post by simion314 » Thu, Sep 02, 2021, 2:35 PM UTC

fallingknife wrote: post #46

eganist wrote: post #37

Earlier quoted context omitted.

"I'm sorry, but how can you form an opinion so quickly?" It doesn't really matter. They're stationary objects; there's absolutely no reason, good or bad, for the algorithm to determine that the correct trajectory change is into the stationary object. Humans crash into stationary objects because they're lacking attention almost 100% of the time, whether it's because they're half asleep, drunk, texting, or distracted b…

There is absolutely a good reason for adaptove cruise control systems to ignore stationary objects: https://arstechnica.com/cars/2018/06/why-emergency-braking-s...

The reason is "our code is too stupid, because is not an actual intelligence and better sensors are expensive"? This is a bad reason from my point of view.

You claim that you have some super duper AI but it can't detect a truck or a wall from a shadow or bag , then your AI is stupid and buy some better sensors and improve your AI.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#85

Post by SerLava » Thu, Sep 02, 2021, 2:37 PM UTC

matt123456789 wrote: post #76

duxup wrote: post #74

Earlier quoted context omitted.

I suspect humans are really bad when they only have to sorta pay attention and it results in just not paying attention a great deal of the time. I'm not sure that's fixable on the human end.

The question is, how much of that is Tesla’s fault? They’re an EV company, not a psychology consultancy. We don’t hold gun manufacturers or alcohol companies liable when people do stupid things with their products, despite the best efforts of their marketing divisions. Granted “pull trigger get bullet” is much simpler than the Autopilot response.

It's not "doing a stupid thing" though

Fundamentally any life form has a slower reaction when they have to notice an issue and start to take motor control. Versus already being in control and simply not causing the issue.

They are not just "an EV company" they are making a guidance system for thousands of high speed metal death machines. They BETTER consult some psychologists. This is a really bizarre thing to say.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#86

Post by yawaworht1978 » Thu, Sep 02, 2021, 2:38 PM UTC

duxup wrote: post #74

yawaworht1978 wrote: post #70

The absolving argument that it's still the drivers responsibility to intervene has flown out of the window with this request. The federal safety agency is not having any of that excuse anymore. They might find that Tesla did no wrong here, further enabling Tesla to get away with this. But this looks a lot like they want to regulate the feature a bit more. If self regulation via companies and humans does not work, the…

I suspect humans are really bad when they only have to sorta pay attention and it results in just not paying attention a great deal of the time. I'm not sure that's fixable on the human end.

It's the counter thesis to the " i drive more alert when drunk" , I know what you mean. Might be that there is no in-between from full self driving and fully humanly operated. Of course, liability under the current state of affairs will always be on the human, but I think it might be reflected in insurance premiums one day where it becomes economically difficult to pay the premium. I agree, all the brands should be observed, Tesla simply has the largest attach surface currently, due to most media exposure and let's be honest, loudest and most misleading marketing strategies.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#87

Post by w-m » Thu, Sep 02, 2021, 2:38 PM UTC

Traster wrote: post #28

Let's ignore the larger point (which is that this is just one of a million possible edge cases where autopilot could systematically fail), I could absolutely see Tesla just bodging in a "If you see emergency vehicle lights flashing disengage" - it'd perfectly reasonable to say that you shouldn't be using autopilot in that situation because you should be paying attention and ready to react to the emergency vehicle.

If the driver isn’t paying attention already, this can’t work in the worst case. The roads and their speed limits are designed for the driver to safely come to a stop if there’s an obstacle in the road. If you add another 3-10 seconds for the system to disengage, warn the user, have the user focus in the road and act appropriately, you are crashed into the emergency vehicle again.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#88

Post by Someone » Thu, Sep 02, 2021, 2:38 PM UTC

capableweb wrote: post #6

meepmorp wrote: post #2

> The safety agency informed Tesla weeks ago that it was looking into the spate of crashes in which Tesla vehicles operating on Autopilot failed to detect stopped emergency vehicles with flashing lights. The regulator originally said it was looking into 11 such crashes. A 12th occurred on Saturday, when a Model 3 hit a police cruiser that had stopped behind a car that had broken down on an interstate in Orlando, Fla.…

> 12 crashes into vehicles with flashing lights on them doesn't sound like great success from Autopilot, gonna be honest. I'm sorry, but how can you form an opinion so quickly? 12 crashes out of what? Any number? 12 crashes out of 10000000 scenarios sound like a pretty good record, while 12 crashes out of 15 scenarios indeed doesn't sound like a great success. But what information do you have that the rest of us does…

It’s not that simple. 12 out of ten million is extremely bad if the average driver takes part in that lottery 100,000 times a day. 12 out of 15 is nothing to worry about if the event is extremely rare.

I guess this lies somewhere between these extremes, but wouldn’t know.

We also don’t know how many of these accidents were prevented by attentive drivers.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#89

Post by fallingknife » Thu, Sep 02, 2021, 2:42 PM UTC

Piskvorrr wrote: post #61

fallingknife wrote: post #38

Earlier quoted context omitted.

You could say the same about airplane autopilots, but they have an amazing safety record.

Far fewer objects to hit in the air: disengagement means "keep going this way," rarely "and crash into an eventual obstacle". Plus, their early safety record is written in blood like everything else is.

> Plus, their early safety record is written in blood like everything else is.

Yeah, and that's how it's going to be with car autopilots. Which is why it pisses me off that people want to stop progress on it.

Re: Tesla is ordered to turn over Autopilot data to a federal safety agency

#90

Post by amelius » Thu, Sep 02, 2021, 2:45 PM UTC

simion314 wrote: post #80

lftl wrote: post #53

Earlier quoted context omitted.

I can't find the specific comment now, but on previous autonomous driving discussions here, it's been asserted that Tesla (and many other systems) simply can't reliably detect stationary objects at speed. If the systems actually did a naive, "that looks like a stationary object, I'm going to avoid it" they'd barely be able to move in a highway setting.

There are systems that can detect stationary objects, Tesla does not want to use those more reliable systems, so they use the programming methodology of tweaking tweaking the code when a bug appears and hope that was the last one. Sure some AI gurus will claim that humans can detect this objects only with 2 eyes but they forget to mention that behind those 2 eyes is a brain an actual intelligence that is aware of the…

> tweaking the code when a bug appears and hope that was the last one

Tesla should also be prohibited from pushing updates. When the software is updated, the code should be tested for at least X thousand miles and X months before a new version may be pushed. Letting an authority do the pushing avoids the temptation of manufacturers to push updates prematurely. It also solves the problem of unexpected changes for the user at random times.

If safety issues are detected, the automobile (or at least the autopilot) should be disabled until the code has been fixed and fully tested.