Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Waymo recalls software after two self-driving cars hit the same truck

cnn.com

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Waymo recalls software after two self-driving cars hit the same truck

#41

Post by Terr_ »

[Recycled from a older submission] Well, I feel kinda vindicated by this news, after previously noting:

> People worry that ways and times [self-driving cars] are unsafe (separate from overall rates) will be unusual, less-predictable, or involve a novel risk-profile.

In this example, having a secretly cursed vehicle configuration is something that we don't normally think of as a risk-factor from human drivers.

_______

As an exaggerated thought experiment, imagine that autonomous driving achieves a miraculous reduction in overall accident/injury rate down to just 10% of when humans were in charge... However of the accidents that still happen, half are spooky events where every car on the road targets the same victim for no discernible reason.

From the perspective of short-term utilitarianism, an unqualified success, but it's easy to see why it would be a cause of concern that could block adoption.

Re: Waymo recalls software after two self-driving cars hit the same truck

#42

Post by camel_gopher »

Terr_ wrote: post #41

[Recycled from a older submission] Well, I feel kinda vindicated by this news, after previously noting: > People worry that ways and times [self-driving cars] are unsafe (separate from overall rates) will be unusual, less-predictable, or involve a novel risk-profile. In this example, having a secretly cursed vehicle configuration is something that we don't normally think of as a risk-factor from human drivers. ______…

Human drivers just have secretly cursed driver configurations.

Re: Waymo recalls software after two self-driving cars hit the same truck

#43

Post by Terr_ »

camel_gopher wrote: post #42
Terr_ wrote: post #41

[Recycled from a older submission] Well, I feel kinda vindicated by this news, after previously noting: > People worry that ways and times [self-driving cars] are unsafe (separate from overall rates) will be unusual, less-predictable, or involve a novel risk-profile. In this example, having a secretly cursed vehicle configuration is something that we don't normally think of as a risk-factor from human drivers. ______…

Human drivers just have secretly cursed driver configurations.

But not systematically, unless someone invents a completely novel tier of outrage-inducing bumper-sticker.

Re: Waymo recalls software after two self-driving cars hit the same truck

#44

Post by [deleted] »

Terr_ wrote: post #41

[Recycled from a older submission] Well, I feel kinda vindicated by this news, after previously noting: > People worry that ways and times [self-driving cars] are unsafe (separate from overall rates) will be unusual, less-predictable, or involve a novel risk-profile. In this example, having a secretly cursed vehicle configuration is something that we don't normally think of as a risk-factor from human drivers. ______…

[deleted]

Re: Waymo recalls software after two self-driving cars hit the same truck

#45

Post by adrians1 »

This is what people don't appreciate when quoting those statistics about how self-driving cars are safer than humans: when a human driver causes an accident, it was because that particular person did something wrong. When a self-driving car handles a situation wrongly that's a big issue, because all the self-driving cars run the same software.

Re: Waymo recalls software after two self-driving cars hit the same truck

#46

Post by skybrian »

adrians1 wrote: post #45

This is what people don't appreciate when quoting those statistics about how self-driving cars are safer than humans: when a human driver causes an accident, it was because that particular person did something wrong. When a self-driving car handles a situation wrongly that's a big issue, because all the self-driving cars run the same software.

Yes, but when the bug was fixed, it was fixed everywhere.

Re: Waymo recalls software after two self-driving cars hit the same truck

#47

Post by hmcq6 »

bobsomers wrote: post #33
pests wrote: post #15

Earlier quoted context omitted.

How do you drive into a solid war if you have lidar though? To say nothing of predictions, the object is where it's at.... where it's at at that moment. You don't need to predict where it's at now... because you know where it's at.

You can't drive if you only use the current "frame" of data as the basis for your decision. Imagine driving on the highway, a comfortable distance behind a lead vehicle. The planning software would want to slam on the brakes without predicting that the blob of sensor data in front of you is going to continue moving forward at highway speeds. That motion prediction enables the planning software to know that the space…

> That motion prediction enables the planning software to know that the space in front of your vehicle will be unoccupied by the time you reach it.

I think we're all on the same page about this part but what's confusing and hilarious is why would the correct answer ever be to drive into an unmoving object?

If they tried to avoid the truck and swerved and hit a different vehicle there would be no confusion here. But the self driving algorithm is effectively committing suicide (Kamikaze). That's novel.

My guess is that the self-driving car was not able to recognize the truck until it was very close and the sudden appearance of the truck is interpreted by the algorithm as if the truck is moving very fast. And the best answer in that case would be to let the truck pass (basically do what the waymo did).

But that means the lidar information about the shape not moving is being deprioritized in favor of the recognized object being calculated to move fast. A situation which could only really occur if a speeding vehicle plowed through a stationary object.

Fantastic solution for avoiding a situation like this -> https://www.youtube.com/watch?v=BbjjlvOxDYk

But a bad solution for avoiding a stationary object.

Re: Waymo recalls software after two self-driving cars hit the same truck

#48

Post by skygazer »

skybrian wrote: post #46
adrians1 wrote: post #45

This is what people don't appreciate when quoting those statistics about how self-driving cars are safer than humans: when a human driver causes an accident, it was because that particular person did something wrong. When a self-driving car handles a situation wrongly that's a big issue, because all the self-driving cars run the same software.

Yes, but when the bug was fixed, it was fixed everywhere.

That phrasing makes it easy to underestimate the astronomical dimensions of edgecase-space, and what necessarily remains unexplored after such a fix.

Re: Waymo recalls software after two self-driving cars hit the same truck

#49

Post by bsaul »

skybrian wrote: post #46
adrians1 wrote: post #45

This is what people don't appreciate when quoting those statistics about how self-driving cars are safer than humans: when a human driver causes an accident, it was because that particular person did something wrong. When a self-driving car handles a situation wrongly that's a big issue, because all the self-driving cars run the same software.

Yes, but when the bug was fixed, it was fixed everywhere.

how do we know for sure that it was fixed ? What are the conditions that are fixed ?

Re: Waymo recalls software after two self-driving cars hit the same truck

#50

Post by AndrewDucker »

esafak wrote: post #18
carbonatedmilk wrote: post #16

One of the best things I've learnt recently is how to apply the zero blame, process improvement approach that (many) air safety regulators take to my own teams. I'd sat through 'five whys' style postmortems before, but it was reading air safety investigation reports that finally got me to understand it and make it a useful part of how we get better at our jobs. By comparison, the way we're investigating and respondin…

Where did you learn about that approach, and how is it different?

Fascinating article here with an example of it: https://asteriskmag.com/issues/05/why-you-ve-never-been-in-a...
Post reply on HN