Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Waymo recalls software after two self-driving cars hit the same truck

cnn.com

Post reply on HN
Print view View original

11–20 of 82 posts

Re: Waymo recalls software after two self-driving cars hit the same truck

#11

Post by kevinventullo »

gdestus wrote: post #8
add-sub-mul-div wrote: post #5

Earlier quoted context omitted.

Yeah, there's nothing akin to a software update that would cause the entire fleet of human drivers to start driving badly or unexpectedly all at once. We also know how to hold individuals accountable for independent accidents. We know we won't get justice when people will inevitably start to get killed by standard corporate greed, incompetence, enshittification.

I imagine this can be remedied by slow rolling non critical updates out so that the entire fleet doesn't get upended by Bobby Tables at once. You could trivially observe the daily change in accidents/collisions/whatever and adjust fire

That remedy already applies to massive profitable services like Google and Facebook, yet they still have outages caused by sloppy configuration pushes.

Re: Waymo recalls software after two self-driving cars hit the same truck

#12

Post by 1000100_1000101 »

add-sub-mul-div wrote: post #5
im3w1l wrote: post #4

This is my biggest fear with self-driving cars. Correlated failures. As a society we are extremely good at dealing with independent accidents. We can calculate very precisely how many people will die of traffic in a given year and we can account for it, we can have insurances, and we can decide exactly how much we are willing to spend to save a life on the margin. But if everything is fine, everything is fine, everyt…

Yeah, there's nothing akin to a software update that would cause the entire fleet of human drivers to start driving badly or unexpectedly all at once. We also know how to hold individuals accountable for independent accidents. We know we won't get justice when people will inevitably start to get killed by standard corporate greed, incompetence, enshittification.

> there's nothing akin to a software update that would cause the entire fleet of human drivers to start driving badly or unexpectedly all at once.

All, no. Enough to make a difference, easy.

  Black Friday in February!  In-Store offer only!
  $99 Playstation 5 to first 100 customers at each Walmart location!
You can bet there will be a significant increase in people driving badly. edit: Make it Taylor Swift tickets, and you can increase the size of the frenzy.

Re: Waymo recalls software after two self-driving cars hit the same truck

#14

Post by leoh »

im3w1l wrote: post #4

This is my biggest fear with self-driving cars. Correlated failures. As a society we are extremely good at dealing with independent accidents. We can calculate very precisely how many people will die of traffic in a given year and we can account for it, we can have insurances, and we can decide exactly how much we are willing to spend to save a life on the margin. But if everything is fine, everything is fine, everyt…

This will have to be priced in to insurance.

Re: Waymo recalls software after two self-driving cars hit the same truck

#15

Post by pests »

Mageek wrote: post #6
Fricken wrote: post #2

A pick-up truck being towed crooked and backwards. Both vehicles failed to read the situation in the same manner. Autonomous vehicles have various redundant systems built-in that can take priority and override false positives. I was previously under the assumption that one of the really important reasons for Lidar is that it can get you closer to an absolute truth about whether something is a solid object, and where…

The article says it: “We determined that due to the persistent orientation mismatch of the towed pickup truck and tow truck combination, the Waymo AV incorrectly predicted the future motion of the towed vehicle.” It was detected, but it predicted the truck would move in a way that it didn’t end up moving.

How do you drive into a solid war if you have lidar though? To say nothing of predictions, the object is where it's at.... where it's at at that moment. You don't need to predict where it's at now... because you know where it's at.

Re: Waymo recalls software after two self-driving cars hit the same truck

#16

Post by carbonatedmilk »

One of the best things I've learnt recently is how to apply the zero blame, process improvement approach that (many) air safety regulators take to my own teams.

I'd sat through 'five whys' style postmortems before, but it was reading air safety investigation reports that finally got me to understand it and make it a useful part of how we get better at our jobs.

By comparison, the way we're investigating and responding to self-driving safety incidents still seems very primitive. Why is that?

Re: Waymo recalls software after two self-driving cars hit the same truck

#17

Post by avalys »

pests wrote: post #15
Mageek wrote: post #6

Earlier quoted context omitted.

The article says it: “We determined that due to the persistent orientation mismatch of the towed pickup truck and tow truck combination, the Waymo AV incorrectly predicted the future motion of the towed vehicle.” It was detected, but it predicted the truck would move in a way that it didn’t end up moving.

How do you drive into a solid war if you have lidar though? To say nothing of predictions, the object is where it's at.... where it's at at that moment. You don't need to predict where it's at now... because you know where it's at.

It wasn’t a solid wall, it was another vehicle.

The brakes don’t respond immediately - you need to be able to detect that a collision is imminent several seconds before it actually occurs.

This means you have to also successfully exclude all the scenarios where you are very close to another car, but a collision is not imminent because the car will be out of the way by the time you get there.

Yes, at some point before impact the Waymo probably figured out that it was about to collide. But not soon enough to do anything about it.

Re: Waymo recalls software after two self-driving cars hit the same truck

#18

Post by esafak »

carbonatedmilk wrote: post #16

One of the best things I've learnt recently is how to apply the zero blame, process improvement approach that (many) air safety regulators take to my own teams. I'd sat through 'five whys' style postmortems before, but it was reading air safety investigation reports that finally got me to understand it and make it a useful part of how we get better at our jobs. By comparison, the way we're investigating and respondin…

Where did you learn about that approach, and how is it different?

Re: Waymo recalls software after two self-driving cars hit the same truck

#19

Post by extua »

carbonatedmilk wrote: post #16

One of the best things I've learnt recently is how to apply the zero blame, process improvement approach that (many) air safety regulators take to my own teams. I'd sat through 'five whys' style postmortems before, but it was reading air safety investigation reports that finally got me to understand it and make it a useful part of how we get better at our jobs. By comparison, the way we're investigating and respondin…

Another comparison with air safety, the disaster risk threshold is high enough to ground vehicles with suspected faults or flaws.

In this case two self-driving cars crashed into another road vehicle because they failed to recognise (in time) which direction it was moving. Waymo should be commended for having voluntarily issued a software recall, but this problem is severe enough that the decision shouldn't really be up to Waymo's good judgement.

Re: Waymo recalls software after two self-driving cars hit the same truck

#20

Post by digitallis42 »

Klaus23 wrote: post #7

Sounds like they were relying solely on their neural network path prediction, which failed when the truck was dragged at an odd angle. A simple lidar moving object segmentation, which doesn't even know what it's looking at but can always spit out reasonable path predictions, would probably have saved them. I think Mobileye is doing something like this, but they release so little data, which is always full of marketin…

It's unlikely to be neutral network based. This sounds like a model prediction failure. You take a mathematical model of car motion: the rear wheels generally don't steer. The front steered wheels can cause the car to drive along an arc. If you want to predict the arc that will be driven, you take the initial starting heading of the vehicle and project forward in time based on your understanding of the vehicles steering angle. For most "driving in lane at velocity" cases, you generally would assume that the vehicle has very little steering angle input.

We're now getting to see where autonomy needs to develop "spider sense": the scene in front of me feels wrong because some element isn't following the expected behavior maybe in ways that can't really be rationalized about, so we'll become much more conservative/defensive when dealing with it.

Post reply on HN