New Study: Waymo is reducing serious crashes and making streets safer

> Building on that, the Collision Avoidance Benchmarking paper presents a novel methodology to evaluate how well autonomous driving systems avoid crashes. The study, which to our knowledge is the first of its kind, introduces a reference model that represents an ideal human state for driving—the response time and evasive action of a human driver that is non-impaired, with eyes always on the conflict (NIEON). Put simply, unlike an average human driver, NIEON is always attentive and doesn’t get distracted or fatigued¹. The data showed that the Waymo Driver outperformed the NIEON human driver model by avoiding more collisions and mitigating serious injury risk in simulated fatal crash scenarios.

(From https://waymo.com/blog/2022/09/benchmarking-av-safety)

AIUI (I'm not on that team), a major challenge is getting good baseline data. Collision reports may not (reliably) capture that kind of data, and it's clearly subjective or often self-reported outside of cases like DUI charges.

[1]https://waymo.com/blog/2025/04/waymo-and-toyota-outline-stra...

I've now had it happen twice that a car will fully blow through an intersection because they know a Waymo will slam on the breaks to avoid a collision. They basically abuse the car's reflexes.

Also in any sort of situation where the Waymo is being very cautious the biggest danger is the impatient people behind the Waymo who will break the law to go out and around it.

> In the case of Waymo, we have some tentative supporting evidence from this and other studies Waymo has run. However, that is still insufficient, even ignoring the lack of audits by non-conflicted parties, to strongly conclude Waymo is safer than a human. The evidence is promising, but it is only prudent to wait for further confirmation.

You are not making a distinction between concluding unsafe and not being able to conclude safe. It is standard practice to not presume safety and that positive evidence of safety must be presented. Failure to demonstrate statistically sound evidence of danger is not proof of safety. Failure to disprove X is not proof of X. This is a very important point to avoid fallacious conclusions on these matters.

To discuss your specific points. Yes, the data is promising, but it is insufficient.

Traffic fatalities occur on the order of 1 per 60-80 million miles. Waymo has yet to reach even one expected traffic fatality yet. They appear to be on track to doing better, but there is not enough data yet.

The reports Waymo present are authored by Waymo. Even the Swiss Re study is in cooperation with Swiss Re, not a independent study by Swiss Re. The studies are fairly transparent, they point to various public datasets, there are fairly extensive public reporting requirements, and Waymo has not demonstrated clear malfeasance, so we can tentatively assume they are “honest”. But we have plenty of examples of bad actors such as Cruise, cigarette companies, VW , etc. who have done end-runs around these types of basic safeguards.

Waymo operational domain is not equivalent to standard human operational domain. They attempt to account for this in their studies, but it is a fairly complex topic with poor public datasets (which is why they cooperated with Swiss Re) so the correctness of their analysis has not been borne out yet. When Waymo incorporates freeways into their public offerings this will enable a less complicated analysis which would lend greater confidence to their conclusions.

Waymo is still in “testing”. As their processes appear to be good, we should assume that their testing procedures are safer than should be expected out of actual deployment or verification procedures. That is not a negative statement. In fact, it would be problematic if their “testing” procedures were less or even equal in safety to their deployment procedures. That is just how testing is. You can and must apply more scrutiny to incomplete systems in use and prevent increased risks especially while under scrutiny otherwise you are almost certainly going to be worse off in deployment where there is less scrutiny. We have yet to see how this will translate out to deployment, so we will need to wait and see if safety while under test will appropriately apply to safety while in release. This is analogous to improved outcomes for patients in medical studies even if they are given the placebo because they just get more care in general while in the study.

Anyways, Waymo appears to be doing as well and honestly as can be determined by a third party observer. I am optimistic about their data and outcomes, but it is only prudent to avoid over-optimism in safety-critical systems and not accept lazy evidence or arguments. High standards are the only way to safety.