U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes • Hacker News Board

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#421

Post by president » Mon, Aug 16, 2021, 11:00 PM UTC

shrimpx wrote: post #414

Animats wrote: post #302

Right. As I've pointed out previously, Tesla seems to be unable to detect sizable stationary obstacles that are partly blocking a lane, especially if they don't look like the rear end of a car. In addition to emergency vehicles, Teslas on autopilot have plowed into freeway barriers and a street sweeper. That's the usual situation for first responders, who usually try to block as little of the road as possible but oft…

Elon keeps warning the world about the impending hyperintelligent AI that will retool human economies, politics, and religions, yet year after year his cobbled-together AI fails at basic object detection.

It's all marketing and image projection.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#422

Post by throwaway0a5e » Mon, Aug 16, 2021, 11:12 PM UTC

jumpkick wrote: post #411

bit_logic wrote: post #385

Earlier quoted context omitted.

I think we need to add a new level 2 assisted driving skills section to driving tests. Level 2 can be safer but it really requires understanding how level 2 works and limitations. For example, when I use level 2 (mine is Honda but applies to other level 2 as well since they mostly share the same vendor), these are the rules I follow: - Car switching in/out of my lane, I manually take over - Tight curve in the freeway…

If level 2 requires such handholding, what’s the point? Seems to me like it just leads to a false sense of security, giving drivers the feeling that they can trust the self-driving system a lot more than they safely can.

It's basically ultra advanced cruise control that can also handle stop and go traffic on a well marked road. That frees the driver up to dedicate more of the situational awareness budget to other things.

If you use it that way it's fine. If you expect it to be as smart as a student driver it's not fine.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#423

Post by theopsguy » Mon, Aug 16, 2021, 11:13 PM UTC

new_realist wrote: post #315

jedberg wrote: post #276

A lot of people in here saying it is not possible to drive safely with partial self driving. I wonder, how many of those people have actually driven a car with autopilot? I have autopilot on my car, and it definitely makes me a better and safer driver. It maintains my distance from the car in front and my speed while keeping me in my lane, so my brain no longer has to worry about those mundane things. Instead I can s…

As someone who has used Autopilot extensively, I can tell you: you only have the illusion of enhanced safety. In reality, parts of your brain have shut down to save energy, and you've lost some situational awareness, but you can't tell that's happened.

If you can’t tell, how do you know this is the case?

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#424

Post by jedberg » Mon, Aug 16, 2021, 11:13 PM UTC

asdff wrote: post #412

jedberg wrote: post #276

A lot of people in here saying it is not possible to drive safely with partial self driving. I wonder, how many of those people have actually driven a car with autopilot? I have autopilot on my car, and it definitely makes me a better and safer driver. It maintains my distance from the car in front and my speed while keeping me in my lane, so my brain no longer has to worry about those mundane things. Instead I can s…

I think that's pretty reckless honestly, you put a lot of faith in the system being able to detect lane markers. Other than that I could see how adaptive cruise control can be nice, but it's also not hard to engage cruise control and fine tune your speed to the conditions by tapping up or down on the buttons on the wheel.

I don't put any faith in it at all actually. That's why I pay attention while it drives, looking for novel situations, which would include self driving errors, so I can correct them.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#425

Post by btilly » Mon, Aug 16, 2021, 11:16 PM UTC

postmeta wrote: post #346

Animats wrote: post #302

Right. As I've pointed out previously, Tesla seems to be unable to detect sizable stationary obstacles that are partly blocking a lane, especially if they don't look like the rear end of a car. In addition to emergency vehicles, Teslas on autopilot have plowed into freeway barriers and a street sweeper. That's the usual situation for first responders, who usually try to block as little of the road as possible but oft…

As this reddit post pointed out, this appears to be a common problem with radar TACC. https://www.reddit.com/r/teslamotors/comments/p5ekci/us_agen... """ These events occur typically when a vehicle is partially in a lane and radar has to ignore a stationary object. This is pretty standard and inherent with TACC + radar. The faster Tesla pushes the vision only stack to all cars after they’ve validated the data, the fa…

These have different causes.

The problem with radar on the ground is that most of what comes to a radar detector is reflections from a stationary world, with relative delays so small as to be undetectable. So the first step in processing is to filter out everything at the speed of that motionary world. All fixed objects therefore disappear, and you are left sorting out moving objects. Which means you now can't detect stationary objects at all.

Tesla has a different problem. They probably don't have depth perception. They therefore have to classify objects, and make educated guesses about where they are relative to the car. Unexpected kinds of objects, or objects in unexpected configurations, fail to be classified and therefore fail to be analyzed.

In principle, Tesla can succeed. After all we don't have binocular vision past 6 meters either. Tesla is improving.

But they haven't yet.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#426

Post by [deleted] » Mon, Aug 16, 2021, 11:16 PM UTC

oaw-bct-ar-bamf wrote: post #389

icelandicmoss wrote: post #369

Earlier quoted context omitted.

I feel like part of the problem with the kind of autopilot crashes you describe here is how inexplicable they are to humans. Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events that are comprehensible to us -- for instance, driver was distracted, or visibility was poor. But when a supposedly 'all-seeing always watching' autopilot drives straight into a large…

In my opinion the underlying assumption autopilots are built with are wrong. It is assumed that the road is free to drive on. Only when the vehicle computer detects a known object on the road that it knows should not be there it is applying brakes or trying to steer around. I would feel safer if the algorithm would assume the negative case as default and only give the „green light“ once it determined that the road is…

[deleted]

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#427

Post by throwaway0a5e » Mon, Aug 16, 2021, 11:19 PM UTC

bob1029 wrote: post #368

jedberg wrote: post #276

A lot of people in here saying it is not possible to drive safely with partial self driving. I wonder, how many of those people have actually driven a car with autopilot? I have autopilot on my car, and it definitely makes me a better and safer driver. It maintains my distance from the car in front and my speed while keeping me in my lane, so my brain no longer has to worry about those mundane things. Instead I can s…

> my brain no longer has to worry about those mundane things I would be terrified to share the road with someone of this mindset. Your vehicle is a lethal weapon when you are driving it around (assisted or otherwise). At no point can someone claim that a tesla vehicle circa today is able to completely assume the duty of driving. You are still 100% responsible for everything that happens in and around that car. You ha…

You're misunderstanding him at best and projecting at worst.

He's saying that he no longer has to worry about those things the same way cruise control lets you not worry about the speedometer needle and dedicate more of your attention budget outside the car.

Of course you can be an idiot and spend it on your cell phone but that's not really a failure mode specific to any given vehicle technology.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#428

Post by elihu » Mon, Aug 16, 2021, 11:25 PM UTC

icelandicmoss wrote: post #369

Animats wrote: post #302

Right. As I've pointed out previously, Tesla seems to be unable to detect sizable stationary obstacles that are partly blocking a lane, especially if they don't look like the rear end of a car. In addition to emergency vehicles, Teslas on autopilot have plowed into freeway barriers and a street sweeper. That's the usual situation for first responders, who usually try to block as little of the road as possible but oft…

I feel like part of the problem with the kind of autopilot crashes you describe here is how inexplicable they are to humans. Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events that are comprehensible to us -- for instance, driver was distracted, or visibility was poor. But when a supposedly 'all-seeing always watching' autopilot drives straight into a large…

Another aspect of unpredictability is that drivers are expected to be alert and vigilant while using ADAS features, but I get the impression that Tesla's implementation sometimes does things that are completely unexpected. Sometimes you might have to react immediately to something you didn't see coming, because you didn't expect the car to suddenly try to steer into a concrete pillar or something.

It's one thing to have to deal with inexplicable behavior from other cars, but to have to deal with inexplicable behavior from your own car seems quite a bit more unnerving.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#429

Post by njarboe » Mon, Aug 16, 2021, 11:30 PM UTC

Anybody have more information about how many, if any, of these wrecks happened after Tesla moved to vision only Autopilot? Maybe this obvious problem has already been mostly fixed with the new system and possibly a main reason for going to full vision in the first place. It seems to me NHTSA need to move a little faster with this quickly changing technology as they are going back to examine crashes that happened starting from Jan, 2018 with this probe.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#430

Post by crote » Mon, Aug 16, 2021, 11:30 PM UTC

nexuist wrote: post #383

bob1029 wrote: post #368

Earlier quoted context omitted.

> my brain no longer has to worry about those mundane things I would be terrified to share the road with someone of this mindset. Your vehicle is a lethal weapon when you are driving it around (assisted or otherwise). At no point can someone claim that a tesla vehicle circa today is able to completely assume the duty of driving. You are still 100% responsible for everything that happens in and around that car. You ha…

This...is entirely the point OP is making. You get more brain power to watch out for the semi jackknifing into you, the car switching lanes without signaling, the truck about to lose a bucket or chair from its bed. This is stuff you may not catch when you're spending your brain power focusing on staying between the lanes and keeping your distance between the car in front. When you automate away the mundane, exception…

The problem is that it isn't automated away.

Self-driving cars will take that stuff over 99% of the time, but the 1% where it screws up is the dangerous part. There are plenty of examples where a self-driving car seemingly randomly seems to go completely haywire, without any obvious reason.

Instead of spending brain power on driving properly, you now have to spend brain power on looking at what you should be doing _and_ checking if the car actually agrees and is doing it.

Staying 100% focused without actually _doing_ anything is incredibly difficult. Many countries intentionally add curves to their highways to keep drivers alert: having a pencil-straight road for hundreds of miles really messes with the human brain.