U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes • Hacker News Board

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#451

Post by jedberg » Tue, Aug 17, 2021, 1:16 AM UTC

theluketaylor wrote: post #447

jedberg wrote: post #443

Earlier quoted context omitted.

I don't drive a Tesla, I use a different autopilot system. The one I use isn't as aggressive as Tesla, so it doesn't exhibit these behaviors but also requires more manual takeovers. Also it uses facial recognition so you don't have to touch the wheel (but you can touch the wheel and steering manually doesn't disengage it, only the brake and gas do).

Since autopilot is a specifically Tesla term for their L2 driver assistance features, you may be better served in this thread referring specifically to the system you have. Most other L2 systems require so much more driver input and attention that many of the incredulous replies are likely assuming you're talking about Tesla specifically. I'm guessing GM Supercruise since it's the only one I'm aware of that uses eye…

I don't mention the one I use because every time I do I get immediate downvotes. But I use the system from Comma.ai which is based on Openpilot, which refers to itself as open source autopilot.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#452

Post by stephencanon » Tue, Aug 17, 2021, 1:17 AM UTC

nexuist wrote: post #383

bob1029 wrote: post #368

Earlier quoted context omitted.

> my brain no longer has to worry about those mundane things I would be terrified to share the road with someone of this mindset. Your vehicle is a lethal weapon when you are driving it around (assisted or otherwise). At no point can someone claim that a tesla vehicle circa today is able to completely assume the duty of driving. You are still 100% responsible for everything that happens in and around that car. You ha…

This...is entirely the point OP is making. You get more brain power to watch out for the semi jackknifing into you, the car switching lanes without signaling, the truck about to lose a bucket or chair from its bed. This is stuff you may not catch when you're spending your brain power focusing on staying between the lanes and keeping your distance between the car in front. When you automate away the mundane, exception…

> You get more brain power to watch out for the semi

That doesn’t help at all if you’re reading a book or playing on your phone, both of which are things I observe Tesla drivers doing pretty often when I’m in the Bay Area.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#453

Post by icelandicmoss » Tue, Aug 17, 2021, 1:21 AM UTC

tshaddox wrote: post #449

icelandicmoss wrote: post #369

Earlier quoted context omitted.

I feel like part of the problem with the kind of autopilot crashes you describe here is how inexplicable they are to humans. Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events that are comprehensible to us -- for instance, driver was distracted, or visibility was poor. But when a supposedly 'all-seeing always watching' autopilot drives straight into a large…

> I feel like part of the problem with the kind of autopilot crashes you describe here is how inexplicable they are to humans. I don't see why these are inexplicable to humans. It's certainly no more difficult to explain than, say, a (non-adaptive) cruise control in a car from 2000 doing the same thing. > Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events th…

Perhaps 'unintuitive' is a better word to convey what I mean -- as in, there isn't an easily understandable (non-technical) narrative chain of events, there's just 'opaque box malfunctioned'. The cruise-control example you give feels a bit different, as CC doesn't claim to include automated collision avoidance, whereas something labelled 'autopilot' does.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#454

Post by sangnoir » Tue, Aug 17, 2021, 1:27 AM UTC

oaw-bct-ar-bamf wrote: post #389

icelandicmoss wrote: post #369

Earlier quoted context omitted.

I feel like part of the problem with the kind of autopilot crashes you describe here is how inexplicable they are to humans. Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events that are comprehensible to us -- for instance, driver was distracted, or visibility was poor. But when a supposedly 'all-seeing always watching' autopilot drives straight into a large…

In my opinion the underlying assumption autopilots are built with are wrong. It is assumed that the road is free to drive on. Only when the vehicle computer detects a known object on the road that it knows should not be there it is applying brakes or trying to steer around. I would feel safer if the algorithm would assume the negative case as default and only give the „green light“ once it determined that the road is…

> Only when the vehicle computer detects a known object on the road that it knows should not be there it is applying brakes or trying to steer around.

The problem with Tesla's "No LIDAR ever, cameras are good enough" approach is that it fails to detect emergency vehicles: they filter out stationary items out of radar signal as noise[1],and Tesla's ML models probably can't reliably identify oblique vehicles and semi trailers as obstacles.

1. Makes sense in isolation: frequent radar returns from roadside and overhead signs would be a pain to deal with

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#455

Post by amanaplanacanal » Tue, Aug 17, 2021, 1:36 AM UTC

tshaddox wrote: post #450

jumpkick wrote: post #411

Earlier quoted context omitted.

If level 2 requires such handholding, what’s the point? Seems to me like it just leads to a false sense of security, giving drivers the feeling that they can trust the self-driving system a lot more than they safely can.

The point is that it makes driving on the highway easier, just like automatic transmissions, power steering, etc. I'm sure early critics of all of these technologies have said things like "but if the driver doesn't need to spend mental and physical energy to that this they will stop paying attention to the road and will be less safe."

You’re sure? Perhaps you could find where somebody actually said that.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#456

Post by kevin_thibedeau » Tue, Aug 17, 2021, 1:37 AM UTC

icelandicmoss wrote: post #369

Animats wrote: post #302

Right. As I've pointed out previously, Tesla seems to be unable to detect sizable stationary obstacles that are partly blocking a lane, especially if they don't look like the rear end of a car. In addition to emergency vehicles, Teslas on autopilot have plowed into freeway barriers and a street sweeper. That's the usual situation for first responders, who usually try to block as little of the road as possible but oft…

I feel like part of the problem with the kind of autopilot crashes you describe here is how inexplicable they are to humans. Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events that are comprehensible to us -- for instance, driver was distracted, or visibility was poor. But when a supposedly 'all-seeing always watching' autopilot drives straight into a large…

It's perfectly explainable. You have a blind machine with an imperfect sensorium trying to describe an elephant. Correct identification is just getting lucky. The layers of ML improve the odds but can never achieve 100%. The whole scheme is playing dice with other people's safety.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#457

Post by jiggawatts » Tue, Aug 17, 2021, 1:41 AM UTC

harles wrote: post #442

diggernet wrote: post #440

Earlier quoted context omitted.

My problem with those crashes is that they are entirely explicable : The car is blind to stationary objects in the road. (My best guess at the logic is they assume that "anything stationary cannot possibly be in the road, right?") To me, that blindness is simply unacceptable. If there is anything in the road, whether identified or not, it should automatically be flagged as a hazard. That flag should only be removed i…

I’m not sure about newer models without radar, but the older ones explicitly discard stationary returns on their radar. As I understand it, without elevation data it can’t know if it’s a bridge you’ll pass under, a soda can in the road, or a stopped car - so just ignore it all. Of course the vision system is supposed to compensate for this, and it performs poorly on objects it doesn’t see often, like emergency vehicl…

The vision system is supposed to be able to determine an accurate depth map based on a combination of stereo vision and depth-from-defocus. I've seen demos of the real-time depth map, and it looks high-resolution and accurate to about 5-10cm.

So, if they have the input data, why is it being ignored by autopilot?

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#458

Post by bob33212 » Tue, Aug 17, 2021, 2:08 AM UTC

thebruce87m wrote: post #417

bob33212 wrote: post #68

Earlier quoted context omitted.

Every year young drivers die because they were inexperienced and didn't realize they were going too fast to too slow for a certain situation. Once full self driving is statistically safer than humans how will you not let people use it? It is like saying you would rather have 10 children die because of bad driving skills rather than 1 child die because they were not paying attention at all times.

Just statistically safer won’t cut it - it will have to me many orders of magnitude safer. Instead of drunk people and mobile phone users dying it will be random accidents that humans would easily have avoided but is some weird edge case for the ML model. It’ll be a cars plowing down kids on trikes on a clear day, all captured in perfect HD on the cars cameras and in the press the next day with the crying driver blam…

So Drunk people don't kill innocent people? Obviously it can't be just barely safer than humans. It need to be a lot safer to change public opinion

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#459

Post by peeters » Tue, Aug 17, 2021, 2:13 AM UTC

sandworm101 wrote: post #337

peeters wrote: post #202

Earlier quoted context omitted.

This is technically true in Ontario (TIL). > 142 (1) The driver or operator of a vehicle upon a highway before turning (...) from one lane for traffic to another lane for traffic (...) shall first see that the movement can be made in safety, and if the operation of any other vehicle may be affected by the movement shall give a signal plainly visible to the driver or operator of the other vehicle of the intention to m…

That's the sort of law I remember. It is considered a failure to communicate your intention rather than a violation per se in every circumstance.

It could be one of those "if a tree falls in the forest" scenarios. If a cop is near enough to see you not signal, he could easily argue that he himself might have been affected by the turn or lane change.

Re: U.S. opens probe into Tesla’s Autopilot over emergency vehicle crashes

#460

Post by oceanghost » Tue, Aug 17, 2021, 2:26 AM UTC

MR4D wrote: post #381

oceanghost wrote: post #318

Earlier quoted context omitted.

The new Toyota RAV4's have this feature-- if you go out of bounds in your lane they beep and the steer5ing wheel gives a bit of resistance. It also reads the speed limit signs and places a reminder in the display. I think it can brake if it detects something in front of it, but I'm not certain.

Many other cars do as well. My main point (perhaps buried more than it should have been) is that by centralizing accident data along with whether an alert went off (or not), and sharing that with all automobile manufacturers can help this process proceed better. Right now the data is highly fragmented and there is not really a common objective metric by which to make decisions to improve models.

I'm sorry, I entirely missed your point.

I agree that would be reasonable and desirable :-)