Even if the cars of the future won't be able to drive on all roads at all time, I don't see how it would matter all that much. Of course there will always be forms of rural driving where the terrain is unknown and there are limited options for sensor fusion available.
But that's not what's important. What's import is DC, LA, and many other metro cities that can't build more roads, but still need population and economic growth. For these cases, we're looking at something entirely different. If self-driving cars ever come to be a large share of the cars on the road, they become and unignorable option to lift the congestion ceiling on growth.
For major thoroughfares, you're looking at something close to a controlled environment. Even if there's rain or snow, you could have precision car location, relative to static beacons, and relative to other cars. There's no question where the lanes are, because all of these extremely major routes have detailed 3D models that all the algorithms have access to. I'm not saying we have that now, but compared to the other resource expenditures in self-driving technology, it's practically a trivial thing to do.
People don't need a car to drive them anywhere at any time. They need a reasonable option to get to work in the places where the jobs are. I would say that we're losing that right now, and self-driving cars are a solution, rain or no rain.