Waymo recalls software after two self-driving cars hit the same truck
1–10 of 82 posts
Re: Waymo recalls software after two self-driving cars hit the same truck
#2A pick-up truck being towed crooked and backwards. Both vehicles failed to read the situation in the same manner.
Autonomous vehicles have various redundant systems built-in that can take priority and override false positives.
I was previously under the assumption that one of the really important reasons for Lidar is that it can get you closer to an absolute truth about whether something is a solid object, and where that hypothetically solid object is relative to the position of the vehicle, regardless of what the classifier thinks it is seeing.
So did the lidar fail to read the solid object, or was the lidar, was it de-prioritized? or was it simply not available as a fallback?
Presumably Radar and proximity sensors were also involved. What were they doing?
This is a fascinating edge case, and I hope to hear about the real reason for the 2 incidents.
Re: Waymo recalls software after two self-driving cars hit the same truck
#3Link should go to the blog post not CNN.
https://waymo.com/blog/2024/02/voluntary-recall-of-our-previ...
Re: Waymo recalls software after two self-driving cars hit the same truck
#4This is my biggest fear with self-driving cars. Correlated failures. As a society we are extremely good at dealing with independent accidents. We can calculate very precisely how many people will die of traffic in a given year and we can account for it, we can have insurances, and we can decide exactly how much we are willing to spend to save a life on the margin.
But if everything is fine, everything is fine, everything is fine, and then all hell breaks lose? We are not as good at dealing with that.
Re: Waymo recalls software after two self-driving cars hit the same truck
#5This is my biggest fear with self-driving cars. Correlated failures. As a society we are extremely good at dealing with independent accidents. We can calculate very precisely how many people will die of traffic in a given year and we can account for it, we can have insurances, and we can decide exactly how much we are willing to spend to save a life on the margin. But if everything is fine, everything is fine, everyt…
Yeah, there's nothing akin to a software update that would cause the entire fleet of human drivers to start driving badly or unexpectedly all at once.
We also know how to hold individuals accountable for independent accidents. We know we won't get justice when people will inevitably start to get killed by standard corporate greed, incompetence, enshittification.
Re: Waymo recalls software after two self-driving cars hit the same truck
#6A pick-up truck being towed crooked and backwards. Both vehicles failed to read the situation in the same manner. Autonomous vehicles have various redundant systems built-in that can take priority and override false positives. I was previously under the assumption that one of the really important reasons for Lidar is that it can get you closer to an absolute truth about whether something is a solid object, and where…
The article says it: “We determined that due to the persistent orientation mismatch of the towed pickup truck and tow truck combination, the Waymo AV incorrectly predicted the future motion of the towed vehicle.”
It was detected, but it predicted the truck would move in a way that it didn’t end up moving.
Re: Waymo recalls software after two self-driving cars hit the same truck
#7Sounds like they were relying solely on their neural network path prediction, which failed when the truck was dragged at an odd angle.
A simple lidar moving object segmentation, which doesn't even know what it's looking at but can always spit out reasonable path predictions, would probably have saved them.
I think Mobileye is doing something like this, but they release so little data, which is always full of marketing bullshit, that it is hard to know what exactly they are working on.
Re: Waymo recalls software after two self-driving cars hit the same truck
#8This is my biggest fear with self-driving cars. Correlated failures. As a society we are extremely good at dealing with independent accidents. We can calculate very precisely how many people will die of traffic in a given year and we can account for it, we can have insurances, and we can decide exactly how much we are willing to spend to save a life on the margin. But if everything is fine, everything is fine, everyt…
Yeah, there's nothing akin to a software update that would cause the entire fleet of human drivers to start driving badly or unexpectedly all at once. We also know how to hold individuals accountable for independent accidents. We know we won't get justice when people will inevitably start to get killed by standard corporate greed, incompetence, enshittification.
I imagine this can be remedied by slow rolling non critical updates out so that the entire fleet doesn't get upended by Bobby Tables at once. You could trivially observe the daily change in accidents/collisions/whatever and adjust fire
Re: Waymo recalls software after two self-driving cars hit the same truck
#9Earlier quoted context omitted.
Yeah, there's nothing akin to a software update that would cause the entire fleet of human drivers to start driving badly or unexpectedly all at once. We also know how to hold individuals accountable for independent accidents. We know we won't get justice when people will inevitably start to get killed by standard corporate greed, incompetence, enshittification.
I imagine this can be remedied by slow rolling non critical updates out so that the entire fleet doesn't get upended by Bobby Tables at once. You could trivially observe the daily change in accidents/collisions/whatever and adjust fire
[deleted]
Re: Waymo recalls software after two self-driving cars hit the same truck
#10Earlier quoted context omitted.
Yeah, there's nothing akin to a software update that would cause the entire fleet of human drivers to start driving badly or unexpectedly all at once. We also know how to hold individuals accountable for independent accidents. We know we won't get justice when people will inevitably start to get killed by standard corporate greed, incompetence, enshittification.
I imagine this can be remedied by slow rolling non critical updates out so that the entire fleet doesn't get upended by Bobby Tables at once. You could trivially observe the daily change in accidents/collisions/whatever and adjust fire
Works great until the problem condition is not evenly distributed in place and time. Imagine that the release goes out in June but can’t handle icy roads; or the release goes out and can’t handle leap years; or it goes to cars in Iowa but has a problem interpreting ocean mist.