Personally, I’d be happy with just automated freeway driving. That seems like an easier problem to solve (no pedestrians, complex traffic controls, cars going the other way or turning across your path, and visibility is usually pretty good). Why not focus on and perfect that first?
While driving on California freeways I have seen: pedestrians, complex traffic controls (law enforcement rerouting traffic or running moving breaks due to incidents), cars going the other way (drunk or confused), and terrible visibility (fog, snow, dust, heavy rain). It's going to take decades before automated freeway driving is perfected. Those edge cases can't just be ignored.
A general enough system would have no edge cases. It is IMO the required solution, and a driver always on the defensive is what we all are, at lease here in South America.
Just following the rules would have killed me years ago. Other drivers are reckless.