The Problem is that it might not be significant safety outside of well suited situations.
This systems expect the user to shim in fastly if something goes wrong, which just can't work reliable given how human brains work.
Furthermore the situations which can make a self driving car crash are not the same which make a human driver crash.
Such crashes can be pretty arbitrary. Random things like advertisement on a bus has the potential to trick of the self driving car. Similar at least some systems are not very good at handling situations where the lanes are unexpected changed e.g. around a construction site. I had been in more than one situation where the lanes markings what completely messed up and only common sense was left to prevent a potential dangerous situation. This systems don't have common sense.
There already was at least on deadly incident where a Tesla crashed for no apparent reason (blaming the dead driver for not taking over fast enough which again isn't realistic at all, through legal maybe valid).
Tesla is supposedly the leader wrt. self driving cars, but people where easily able to trick that car to drive into the lane for the opposite direction and it has since a while one of the worst emergency brake assistants.
The problem boils down to self driving cars not being good at "unusual" situations but situations which are unusual for the self driving car are in practice pretty common given how much people drive cars and they are not necessary recognizable as such by humans.
Another problem is that traffic's in different countries have different rules and dynamics a self driving system working well for idk. the US might not at all be safe to use in idk. insert arbitrary EU country here even if trained for that country.
Humans can normally adapt to a degree in the fly, which makes them not have a crash, or less bad crashes in some situations. But the self driving car has no such common sense and might not even realize it's in a situation it's not trained for.
I see a lot of potential to make traffic more safe.(non stuff driving but assistant systems, self driving in traffic jams, slow self driving buses in car reduced inner cities, on highway self driving truck convoys, etc.).
But at least on the close future I'm not convinced "general purpose"/"everywhere any speed" self driving cars are more safe.
I mean we also shouldn't forget that a lot of deadly accidents happen because of negligence, so while self driving cars still require a driver hand over, they still all happen because the driver is as negligent (or might intentionally not use the self driving feature to get faster to work etc.).
Edit: Swipy keyboard mess ups.